Background
Artea, an online retailer known for its handmade clothing, experiences high visitor engagement but struggles with a low conversion rate: 87% of visitors do not purchase. To better understand customers and their purchase behaviour, the company ran an A/B test. The team randomly selected half of 5,000 users who had visited the website in the last 2 months but had not made a transaction, sent them a coupon that offered a 20% discount on the next purchase. The coupon was non-transferable to other users and was valid for one month. One month later, Artea marketing team collected the number of transactions and total expenditure made by all of the relevant users.
Task
Our case study task is to analysis the Coupon A/B test campaign and answer the following questions:
In assessing the randomness of the coupon assignment during the A/B test, we compared the test group (with coupon) and the control group (without coupon) across several variables: acquisition channel, browsing minutes, weeks since last visit, and the number of past purchases. The histograms and bar charts demonstrate an even distribution between the two groups for each variable.
Upon sign up, users indicate their gender, race and ethnicity, providing Artea demographical information that could be used to make the targeting decision. At first, CEO Alex hesitated to use demographic data for decision-making, fearing it could introduce biases. She valued equal treatment for all customers but she was aware of algorithms could create biases that were translated into marketing practices.
We assessed whether female or minority customers differed from others in past behaviour among the recent 6,000 visitors to Artea.
From the data file, we found out the following insights.
• Non-males are more likely to have items in the shopping cart.
• Minorities are highly underrepresented in all channels but Google
• Both minority and non-minority groups are equally likely to have items in their shopping cart. There is a similar level of engagement in terms of placing products into the cart between the two groups.
• Minority groups mainly live in states 3, 7, and 10, so they aren't as widely represented. Also, people living in states 1 and 2 are most favoured to receive coupons.