Using Adaptive Experimentation To Design Intelligent Interventions for Belief & Behaviour Change
Prof. Joseph Jay Williams
Assistant Professor in Computer Science
Intelligent Adaptive Interventions Lab
University of Toronto
Businesses, society and individuals struggle to change people’s beliefs and behaviour. This can cost billions in waste, bad physical and mental health outcomes, and poor training. I give examples of how my research redesigns everyday technology to create Intelligent Interventions. I use human & generative AI co-design to generate interventions, and multi-armed bandit algorithms (RL) to evaluate and deploy these interventions. Our approach uses Adaptive A/B experimentation, for which we won the $1M Xprize challenge and a $3M NSF grant. Our interventions impacted beliefs and behaviour in more than 500 000 people, in applications from emails (like AI for sales & marketing), SMS messages to encourage physical exercise, chatbots for mental wellbeing, and personalized digital learning resources.
My statistics research program tackles the problem of bandit algorithms increasing false positives and negatives in Adaptive A/B Experiments, by developing new Statistically Sensitive Algorithms – that maximize user outcomes and increase statistical power – and Algorithm Attuned statistical tests – that control false positive rate.
This research program creates Intelligent Interventions by integrating tools for Generative AI & Adaptive Experimentation, insights for belief and behaviour change, and better methods (algorithms and statistical tests).
An overview of the research program is at tiny.cc/williamsresearch, papers at tiny.cc/williamspapers & tiny.cc/williamsjournalpapers, and slides & recordings from past talks at tiny.cc/williamstalk.
Joseph Jay Williams is an Assistant Professor at the University of Toronto in Computer Science, with courtesy appointments supervising PhD students in Statistical Sciences, Psychology, and the Vector Institute for Artificial Intelligence. He also has courtesy appointments in Economics, Industrial Engineering, & the Faculty of Information. He directs the Intelligent Adaptive Interventions lab, which aims to transform any user interface into an Intervention to help people change their behaviour and learn, by reimagining randomized “A/B” experiments as a tool for Intelligent Adaptation. His lab’s work is represented in over 85 papers ( tiny.cc/williamsjournalpapers), 2 Best Paper Awards (1 at CHI), 4 Runner Up/Honorable Mention for Best Paper (CHI, EDM, LAS), and 1st place in a $1M Xprize competition for the future of experimentation technology. He’s received over $2M in grant funding, enabling interventions impacting over 500 000 people. His PhD Students span HCI (Human Computer Interaction), Cognitive/Social/Clinical/Health Psychology, applied ML (reinforcement learning), applied AI (LLMs), & Statistics. Joseph is originally from Trinidad and Tobago, did his PhD at UC Berkeley, postdoc at Stanford, and was a Research Scientist at Harvard. He was previously an Assistant Professor at the National University of Singapore, in Information Systems & Analytics.
