“Word-of-AI” and Matching Quality: Evidence from a Natural Experiment on Online Review Platforms
Professor Xitong Li
Professor of Information Systems
Department of Information Systems and Operations Management
HEC Paris
ABSTRACT
Many online platforms have recently integrated generative AI (GAI) generated content, yet its impact on the platform ecosystem requires careful investigation. By leveraging a unique policy of a leading online review platform that introduces GAI reviews summary (GAIRS), this study examines how GAIRS can affect the matching quality of online consumers purchasing products or services. Constructing a unique panel dataset of online reviews for a set of hotels on both TripAdvisor and Expedia, we apply a cross-platform difference-in-differences approach to estimate the impact of GAIRS. Our findings elucidate the positive effects of GAIRS on matching quality, manifested by increased consumer rating and decreased rating dispersion. This effect is driven by a decrease in unsatisfactory consumer experiences. Exploring potential mechanisms, we show that the positive effect of GAIRS on matching quality is more prominent in hotels with high uncertainty and when GAIRS is generated from a larger number of reviews, contains more content, or exhibits greater readability. We also present direct evidence supporting our mechanism by showing that the consumer reviews post-GAIRS display greater certainty and assertiveness in their content. Our further analyses rule out an alternative explanation for GAIRS’s role being a form of top review, by showing evidence for the performance of solicited reviews, the absence of consumer imitation from GAIRS, and improvements in hotel performance. Finally, we employ transfer deep learning to further demonstrate that GAIRS can reduce uncertainty. Additionally, we find that improvements in experiential dimensions including rooms, value, noise level, and service drive the decline in unsatisfactory consumer experiences. This research highlights the potential of GAIRS, as a recent GAI-empowered application in online platforms, in improving matching between online consumers and products, thereby contributing to the expanding discourse on the impacts of GAI in online markets.