25/04/2020

Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems

C. Estelle Smith, Bowen Yu, Anjali Srivastava, Aaron Halfaker, Loren Terveen, Haiyi Zhu

Keywords: wikipedia, peer production, value sensitive algorithm design, machine learning, ores, community values

Abstract Paper Similar Papers

Abstract: On Wikipedia, sophisticated algorithmic tools are used to assess the quality of edits and take corrective actions. However, algorithms can fail to solve the problems they were designed for if they conflict with the values of communities who use them. In this study, we take a Value-Sensitive Algorithm Design approach to understanding a community-created and -maintained machine learning-based algorithm called the Objective Revision Evaluation System (ORES)—a quality prediction system used in numerous Wikipedia applications and contexts. Five major values converged across stakeholder groups that ORES (and its dependent applications) should: (1) reduce the effort of community maintenance, (2) maintain human judgement as the final authority, (3) support differing peoples’ differing workflows, (4) encourage positive engagement with diverse editor groups, and (5) establish trustworthiness of people and algorithms within the community. We reveal tensions between these values and discuss implications for future research to improve algorithms like ORES.

The video of this talk cannot be embedded. You can watch it here:
https://www.youtube.com/watch?v=Dn0-34jFK2Y
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at CHI 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 13:37 
 0
 0
 0
 0
 13:30 
 0
 0
 0
 0
 12:13 
 0
 0
 0
 0
 5:22 
 0
 0
 0
 0
 5:12 
 0
 0
 0
 0
 14:37 
 0
 0
 0
 0
 12:20 
 0
 0
 0
 0
 11:49 
 0
 0
 0
 0
 8:54 
 0
 0
 0
 0
 13:20 
 0
 0
 0
 0
 2:28 
 0
 0
 0
 0
 11:41 
 0
 0
 0
 0
 18:36 
 0
 0
 0
 0
 10:00 
 0
 0
 0
 0
 14:33 
 0
 0
 0
 0
 18:05 
 0
 0
 0
 0
 14:16 
 0
 0
 0
 0
 16:16 
 0
 0
 0
 0
 10:00 
 0
 0
 0
 0
 8:40 
 0
 0
 0
 0
 4:44 
 0
 0
 0
 0
 21:26 
 0
 0
 0
 0
 11:47 
 0
 0
 0
 0
 8:26 
 0
 0
 0
 0
 12:19 
 0
 0
 0
 0
 8:04 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 6:32 
 0
 0
 0
 0
 8:04 
 0
 0
 0
 0
 3:08 
 0
 0
 0
 0
 15:00 
 0
 0
 0
 0
 19:22 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 12:53 
 0
 0
 0
 0
 3:16 
 0
 0
 0
 0
 15:03 
 0
 0
 0
 0
 13:58 
 0
 0
 0
 0
 11:54 
 0
 0
 0
 0
 9:58 
 0
 0
 0
 0
 14:28