Abstract: Cognitive tasks are increasingly being gamified in an attempt to leverage the motivational power of games; however, they are sensitive to manipulation and literature is divided on how adding game elements affects participant performance and experience. We applied two popular gamification approaches (points/feedback and theme/narrative) to a typical cognitive task (the dot probe) and measured performance and experience in two studies (N1=287, N2=321). Similar to prior work, we confirm in Study1 that points increase reaction time and error rate, and positive affect. We replicated these results in Study2, and expanded our analysis to investigate participant experience. Our findings suggest that theme creates expectations of an interesting game, which gamified tasks fail to deliver, whereas points maintain enjoyment better throughout the task itself. Important for the development of gamified cognitive tasks, our findings suggest that novel approaches to gameful assessment may be better than the status quo.