Abstract:
A lack of corpora has so far limited advances in integrating human gaze data as a
supervisory signal in neural attention mechanisms for natural language processing
(NLP). We propose a novel hybrid text saliency model (TSM) that, for the first
time, combines a cognitive model of reading with explicit human gaze supervision
in a single machine learning framework. On four different corpora we demonstrate
that our hybrid TSM duration predictions are highly correlated with human gaze
ground truth. We further propose a novel joint modeling approach to integrate TSM
predictions into the attention layer of a network designed for a specific upstream
NLP task without the need for any task-specific human gaze data. We demonstrate
that our joint model outperforms the state of the art in paraphrase generation on
the Quora Question Pairs corpus by more than 10% in BLEU-4 and achieves
state of the art performance for sentence compression on the challenging Google
Sentence Compression corpus. As such, our work introduces a practical approach
for bridging between data-driven and cognitive models and demonstrates a new
way to integrate human gaze-guided neural attention into NLP tasks.
