Modeling Human Visual Search Performance on Realistic Webpages Using Analytical and Deep Learning Methods

Abstract: Modeling visual search not only offers an opportunity to predict the usability of an interface before actually testing it on real users but also advances scientific understanding about human behavior. In this work, we first conduct a set of analyses on a large-scale dataset of visual search tasks on realistic webpages. We then present a deep neural network that learns to predict the scannability of webpage content, i.e., how easy it is for a user to find a specific target. Our model leverages both heuristic-based features such as target size and unstructured features such as raw image pixels. This approach allows us to model complex interactions that might be involved in a realistic visual search task, which can not be achieved by traditional analytical models. We analyze the model behavior to offer our insights into how the salience map learned by the model aligns with human intuition.

06/12/2020

Algorithms -> Image Segmentation; Algorithms -> Semi-Supervised Learning; Applications -> Computer Vision; Applications -> Imag, Algorithms -> Adversarial Learning

3:25

22/11/2021

Benjamin Sanchez-Lengeling, Jennifer Wei, Brian Lee and
Emily Reif, Peter Wang, Wesley Qian, Kevin McCloskey, Lucy Colwell , Alexander Wiltschko

biologically plausible, translation invariance, robustness, sequential vision, transsaccadic vision, grid cells, path integration, continual learning, predictive representations, Hebbian learning

11:21

19/10/2020

MTBRN: Multiplex target-behavior relation enhanced network for click-through rate prediction

Yufei Feng, Fuyu Lv, Binbin Hu and
Fei Sun, Kun Kuang, Yang Liu, Qingwen Liu, Wenwu Ou

inference in crf, semantic segmentation, structured prediction, learning to optimize, reinforcement learning, graph neural network, markov chain tree search, deep q-network, graphical models, combinatorial optimization

3:59

12/07/2020

Abhishek Das, Federico Carnevale, Hamza Merzic and
Laura Rimell, Rosalia Schneider, Josh Abramson, Alden Hung, Arun Ahuja, Stephen Clark, Greg Wayne, Feilx Hill

Sean Augenstein, H. Brendan McMahan, Daniel Ramage and
Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, Blaise Aguera y Arcas

medical image segmentation, selective inference, post-selection inference, startistical significance, graph-cut segmentation

0:58

06/12/2020

Tianmin Shu, Abhishek Bhandwaldar, Chuang Gan and
Kevin Smith, Shari Liu, Dan Gutfreund, Elizabeth Spelke, Josh Tenenbaum, Tomer Ullman

Sam Sattarzadeh, Mahesh Sudhakar, Anthony Lem and
Shervin Mehryar, Konstantinos N Plataniotis, Jongseong Jang, Hyunwoo Kim, Yeonjeong Jeong, Sangmin Lee, Kyunghoon Bae