For Bamako.ai, our flagship event, we needed to launch a massive messaging campaign to motivate our community to attend. As an evidence-guided organization, we did what we do best: we designed a quick and dirty experiment to evaluate different message variants.
Experimental design
We utilized two data sources: the WhatsApp API for read information and Amplitude for website visit and registration data. We divided our audience into two groups:
- G1: Participants from last year’s Bamako.ai who do not have a Kabakoo app account
- G2: Non-participants from last year who have a Kabakoo app account
Our hypothesis concerning the two groups was that having an account in the app would lead to a higher conversion rate compared to having participated in last year’s event. We were expected this since having an active account in the app should be associated to a stronger bond with the Kabakoo community.
We crafted four distinct messages, each leveraging different psychological biases. Our choice of biases was informed by the work at the Busara Center and our previous experiences:
- M1: Urgency — Emphasized the approaching date with the phrase “Les places partent vite, inscris-toi sans tarder!” (Spots are filling up fast, register without delay!)
- M2: Tangibility and Identity — Highlighted projects from last year’s Bamako.ai, plus suggesting that recipients could create similar projects.
- M3: Positive Storytelling — Featured Mamadou, a participant from last year who successfully launched his project thanks to Bamako.ai.
- M4: Urgency with Anchoring — Another take on urgency, but included a specific amount that could potentially be won.
Each message was accompanied by the event flyer as an attached image.
We sent 8,000 WhatsApp messages in the first stage, with each message variant sent to 2,000 recipients (1,000 from G1 and 1,000 from G2). Every message included a “S’inscrire!” (Register) button at the bottom, which directed recipients to the Bamako.ai landing page upon clicking.
Our primary metric was the conversion rate, defined as the proportion of people who read the message and subsequently registered to the event on the Bamako.ai webpage.
Results
The chart below illustrates how first-level conversion rates varied across groups and messages. Group 2 consistently showed higher conversion rates compared to Group 1, supporting our hypothesis that being a registered app member increases engagement more than past event participation.
Among the message variants, M2 (tangibility and identity) consistently outperformed others in both groups.
Combining both groups, M2 (anchoring & identity) emerged as the clear winner with a 16.27% first-level conversion rate, followed by M4 (urgency and anchoring) at 14.71%.
We also analyzed registration data, which confirmed M2’s superior performance. The following funnel provides a detailed breakdown of each message’s performance:
As displayed by the funnel above, M2 shows a registered/read conversion rate of 8.9% while M3 and M4 are around 6.5% and M1 even at 5.3%.
Next steps
Armed with these insights, we used the winning variant, M2, for the remaining messages in our campaign. As a reminder, M2 focused on tangibility and identity, highlighting projects from last year’s Bamako.ai and encouraging recipients to envision themselves creating similar projects.
We acknowledge that using two different biases in the same message (tangibility and identity) can confound results and complicate interpretation. Despite this limitation, we were eager to experiment and compare outcomes with last year’s smaller-scale experiment.
This quick messaging experiment for Bamako.ai underscores the importance of experimentation in optimizing campaigns. By adopting an evidence-guided approach, we successfully identified the most effective message variant and target audience, ultimately boosting attendance for our flagship event.
Are you familiar with running similar experiments? We would love to hear from you!
