Method

Evidence you can
trust, or challenge

Kabakoo's evaluation system was designed with Prof. Dr. Martin Schneider, Chair of Personnel Economics at the University of Paderborn, and his team. The core design is a recruit-and-delay RCT that became quasi-experimental through capacity-driven crossovers. Dr. Jamie Walsh, Senior Behavioral Scientist, and Elia Gandolfi, Data Scientist at The Agency Fund, independently run their own analyses on our data.

Tier 1 | Experimental and panel evidence

The 32-month evaluation panel

Sample
Treatment n=226, clean control n=55 (pilot evaluation cohort, baseline January 2022)
Design
Recruit-and-delay RCT, became quasi-experimental through capacity-driven crossovers; baseline-adjusted estimates
Time horizon
32 months
Partners
University of Paderborn (Schneider, Herdejürgen, Islam, Kabst); independent partial replication by Jamie Walsh, The Agency Fund
Status
Findings published in Kabakoo's April 2026 evidence brief.
Link
Evidence brief PDF and replication package via research@kabakoo.africa

What it found

OutcomeResultSource
Share earning ≤40,000 FCFA/month46% (control) vs 38% (treatment)32-month panel | July 2026 analysis round
Share earning >250,000 FCFA/month7% (control) vs 11% (treatment)32-month panel | July 2026 analysis round
Growth mindset+0.14 to +0.25 SD32-month panel | University of Paderborn evaluation | April 2026
Continued skills training+0.36 SD (p=0.004)32-month panel | University of Paderborn evaluation | April 2026
Earnings as an employee+0.25 SD (p=0.029)32-month panel | University of Paderborn evaluation | April 2026
Made no professional investment5% of treated vs 14% of clean controls (p=0.005)32-month panel | University of Paderborn evaluation | April 2026
Kabakoo Behavioral Index+0.33 behaviors on a 0–6 scale (p=0.077)32-month panel | University of Paderborn evaluation | April 2026
Net Promoter Score61 → 77First survey vs 32-month follow-up, n=358/150 | April 2026
Return per dollar invested, including spillovers 5–8x

5.1–8.4x spillover-adjusted at 32 months | University of Paderborn evaluation | July 2026

Counting measured spillovers, each dollar invested returns 5.1–8.4x at 32 months. The direct-only floor is 3–5x, computed conservatively, with self-reported income gains cut by 25% and the full per-participant cost, 10% discount rate.

Spillovers, observed. Non-participants passively exposed to Kabakoo learners earned +26,700 FCFA (~$43) more in the prior month; actively exposed peers, +47,800 FCFA (~$78), with +18.6 percentage points on earning as an employee. 

The 2022 pilot RCT

Sample
n=116
Design
Randomized controlled trial
Time horizon
Program period, 2022
Partners
University of Paderborn
Status
Published: Islam, Herdejürgen, Kemayou, Kabst & Schneider (2023), "From obstacle to opportunity: How growth mindsets foster employability & entrepreneurship in Mali," Academy of Management Proceedings
What it found
Growth mindset +23% (p=0.03). Superseded by the 32-month panel estimates above.
Link
DOI 10.5465/AMPROC.2023.19425abstract

BEAT Covid-19, four years on

Sample
29 participants (EU/EJOM-funded, 2020)
Design
Long-horizon alumni follow-up with a comparison group; small n
Time horizon
4 years
Partners
EU Delegation to Mali (program funding)
Status
Reported May 2025
What it found
4.9x ROI; 83.4% of alumni earn over 150,000 FCFA/month vs 30% in the comparison group. 
Link
research@kabakoo.africa
Tier 2 | Quasi-experimental comparisons and A/B tests

Kabakoo runs randomized product experiments as routine operations. The publishable inventory, each with its honest label:

Digital Makers Cohort 3, twelve months on

Sample
Longitudinal sample of 80 (36 participants, 44 controls)
Design
Recruit-and-delay comparison, tracked over time
Time horizon
Twelve months post-program (T2)
Status
Published, March 2026 monthly update
What it found
66.7% of participants employed vs 52.3% of controls; Kabakoo Behavioral Index 4.19 vs 3.80; NPS rising after the program ends (70 → 81 in earlier cohorts, 58 → 75 in this one). The same signals keep repeating across cohorts
Link
research@kabakoo.africa

WLS cohort 1 engagement, analyzed independently

Sample
826 learners enrolled in the first WhatsApp-only cohort
Design
Independent analysis by Elia Gandolfi, Data Scientist and Product Lead at The Agency Fund (June 2026)
What it found
412 learners (50%) never took a single action. Week-0 activity of 84% fell to 29% by week one. The 73 learners with at least one in-person touchpoint sent 4x more messages and watched 48x more module videos than digital-only learners (Mann-Whitney p<0.001, r=0.52). Grit, growth mindset, and self-efficacy did not separate active from inactive learners (p=0.963, 0.706, 0.757); prior social connection did
Status
Published, June 2026 monthly update; comparisons are observational, not randomized

AI-generated vs human tutorial video

Sample
n=414 / 413 assigned
Design
Randomized A/B, March 6–11, 2026
What it found
First key action within 24h: 14.4% (AI video) vs 11.0% (human), +31% relative. Conversation depth identical (6.93 vs 6.91 messages). The AI video won on activation and changed nothing deeper
Status
Published, April 2026 monthly update

WhatsApp nudges vs push notifications

Sample
N=271, randomized
What it found
WhatsApp nudges lifted activation (≥2 logins) by 15.6%. No effect on deeper engagement. 
Status
Published, February 2026 monthly update

Replication

Check our work

The anonymized panel, analysis scripts, codebooks, and a  replication pipeline exist and are shared with research partners. Contact us at research@kabakoo.africa. 

Request the replication package