Control Group Representivity

If your goal is to measure overall marketing effectiveness, GCG is more reliable. If you need to optimize individual campaigns and channels, Canvas-Level Controls are better.

Would you like a recommendation on a hybrid approach? (e.g., GCG for broad measurement, CCG for testing tactics within active users).

Hey NatalieMc,

Before the out-of-the-box Braze Universal Control Group (UCG), I used a similar solution to your canvas one (I suspect) but through a Webhook campaign: new users would enter the webhook, we would assign them to the UCG randomly, based on the random buckets, and then we'd log a custom attribute in their profile to identify whether UCG was true/false.

With the current Braze Universal Control Group, the logic is similar but just more straightforward to setup/maintain. As far as I know, Braze also uses the random buckets to assign a random sample of your base to the UCG so it should be representative.

I think the latter is simpler, because not only you can handpick the % you want to allocate, but it's much easier to maintain and to ensure they are excluded from all campaigns, without relying on each team member to remember to add that filter to targeting every time. At the same time, you still have the flexibility to add "exclusions" and ensure that campaigns meeting certain conditions can still override the UCG (for example, transactional or key onboarding messages). You can easily pass the list of UCG members to other tools for reporting as well.

Overall, one good practice to keep in mind is that it might be worth re-setting your UCG every X periods of time. That's because if UCG users are never eligible to receive any messages, over time, they will naturally have a biased behavior and thus be less "comparable" to the treatment group. In the documentation, there're some useful tips on calculating size, things to watch out for, and experiment duration.

Hope that helps? :) good luck

Yes, it does use Random Bucket numbers! Docs are here: https://www.braze.com/docs/user_guide/engagement_tools/testing/global_control_group/#how-the-global-control-group-works

Braze randomly selects multiple ranges of random bucket numbers and includes users from those selected buckets.