Knowledge Graph Enhanced Neural Machine Translation via Multi-task Learning on Sub-entity Granularity

Abstract: Previous studies combining knowledge graph (KG) with neural machine translation (NMT) have two problems: i) Knowledge under-utilization: they only focus on the entities that appear in both KG and training sentence pairs, making much knowledge in KG unable to be fully utilized. ii) Granularity mismatch: the current KG methods utilize the entity as the basic granularity, while NMT utilizes the sub-word as the granularity, making the KG different to be utilized in NMT. To alleviate above problems, we propose a multi-task learning method on sub-entity granularity. Specifically, we first split the entities in KG and sentence pairs into sub-entity granularity by using joint BPE. Then we utilize the multi-task learning to combine the machine translation task and knowledge reasoning task. The extensive experiments on various translation tasks have demonstrated that our method significantly outperforms the baseline models in both translation quality and handling the entities.

04/07/2020

Knowledge Graph Enhanced Neural Machine Translation via Multi-task Learning on Sub-entity Granularity

Yang Zhao, Lu Xiang, Junnan Zhu, Jiajun Zhang, Yu Zhou, Chengqing Zong

Comments

Similar Papers

Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings

Apoorv Saxena, Aditay Tripathi, Partha Talukdar

Keywords Abstract Paper

Multi-hop Answering, Question task, natural queries, multi-hop KGQA

Multi-source Meta Transfer for Low Resource Multiple-Choice Question Answering

Ming Yan, Hao Zhang, Di Jin, Joey Tianyi Zhou

Keywords Abstract Paper

Multi-source Transfer, Low Answering, Multiple-choice answering, machine comprehension

Data Augmentation for Abstractive Query-Focused Multi-Document Summarization

Ramakanth Pasunuru, Asli Celikyilmaz, Michel Galley and Chenyan Xiong, Yizhe Zhang, Mohit Bansal, Jianfeng Gao

Keywords Abstract Paper

Mirror-Generative Neural Machine Translation

Zaixiang Zheng, Hao Zhou, Shujian Huang and Lei Li, Xin-Yu Dai, Jiajun Chen

Keywords Abstract Paper

neural machine translation, generative model, mirror

LoCo: Local Contrastive Representation Learning

Yuwen Xiong, Mengye Ren, Raquel Urtasun

Keywords Abstract Paper

Unsupervised Domain Adaptation with Joint Domain-Adversarial Reconstruction Networks

Qian Chen, Yuntao Du, Zhiwen Tan and Yi Zhang, Chongjun Wang

Keywords Abstract Paper

unsupervised domain adaptation, domain-adversarial learning, data reconstruction, distribution alignment

DecAug: Out-of-Distribution Generalization via Decomposed Feature Representation and Semantic Augmentation

Haoyue Bai, Rui Sun, Lanqing Hong and Fengwei Zhou, Nanyang Ye, Han-Jia Ye, S.-H. Gary Chan, Zhenguo Li

Keywords Abstract Paper

Jointly non-sampling learning for knowledge graph enhanced recommendation

Chong Chen, Min Zhang, Weizhi Ma and Yiqun Liu, Shaoping Ma

Keywords Abstract Paper

recommender systems, non-sampling learning, knowledge graph, implicit feedback, efficient

Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learning

Kibeom Kim, Min Whoo Lee, Yoonsung Kim and JeHwan Ryu, Minsu Lee, Byoung-Tak Zhang

Keywords Abstract Paper

reinforcement learning and planning

Uncertainty-Aware Semantic Augmentation for Neural Machine Translation

Xiangpeng Wei, Heng Yu, Yue Hu and Rongxiang Weng, Luxi Xing, Weihua Luo

Keywords Abstract Paper

sequence-to-sequence task, nmt, inference, translation tasks

Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data

Jonathan Pilault, Amine EL hattami, Chris J Pal

Keywords Abstract Paper

Natural Language Processing, Transfer Learning, Adaptive Learning, Multi-Task Learning

Benchmarking Meaning Representations in Neural Semantic Parsing

Jiaqi Guo, Qian Liu, Jian-Guang Lou and Zhenwen Li, Xueqing Liu, Tao Xie, Ting Liu

Keywords Abstract Paper

meaning representation, semantic parsing, unimer, meaning representations

It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information

Emanuele Bugliarello, Sabrina J. Mielke, Antonios Anastasopoulos and Ryan Cotterell, Naoaki Okazaki

Keywords Abstract Paper

Measuring Difficulty, generation, asymmetric difficulty, machine difficulty

Knowledge-aware Leap-LSTM: Integrating Prior Knowledge into Leap-LSTM towards Faster Long Text Classification

Jinhua Du, Yan Huang, Karo Moilanen

Keywords Abstract Paper

Multi-Task Learning for Knowledge Graph Completion with Pre-trained Language Models

Bosung Kim, Taesuk Hong, Youngjoong Ko, Jungyun Seo

Keywords Abstract Paper

TempLe: Learning Template of Transitions for Sample Efficient Multi-task RL

Yanchao Sun, Xiangyu Yin, Furong Huang

Keywords Abstract Paper

An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels

Ilias Chalkidis, Manos Fergadiotis, Sotiris Kotitsas and Prodromos Malakasiotis, Nikolaos Aletras, Ion Androutsopoulos

Keywords Abstract Paper

flat classification, hierarchical approaches, zero-shot learning, few learning

Imbalance Robust Softmax for Deep Embedding Learning

Hao Zhu, Yang Yuan, Guosheng Hu and Xiang Wu, Neil Robertson

Keywords Abstract Paper

Zero-Resource Cross-Domain Named Entity Recognition

Zihan Liu, Genta Indra Winata, Pascale Fung

Keywords Abstract Paper

Large-Scale Cross-Domain Few-Shot Learning

Jiechao Guan, Manli Zhang, Zhiwu Lu

Keywords Abstract Paper

Low-Rank Bottleneck in Multi-head Attention Models

Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat and Sashank Jakkam Reddi, Sanjiv Kumar

Keywords Abstract Paper

Deep Learning - Theory

Structured Reordering for Modeling Latent Alignments in Sequence Transduction

Keywords Paper

Keywords Paper

Ramakanth Pasunuru, Asli Celikyilmaz, Michel Galley and
Chenyan Xiong, Yizhe Zhang, Mohit Bansal, Jianfeng Gao

Keywords Paper

Zaixiang Zheng, Hao Zhou, Shujian Huang and
Lei Li, Xin-Yu Dai, Jiajun Chen

Keywords Paper

Keywords Paper

Qian Chen, Yuntao Du, Zhiwen Tan and
Yi Zhang, Chongjun Wang

Keywords Paper

Haoyue Bai, Rui Sun, Lanqing Hong and
Fengwei Zhou, Nanyang Ye, Han-Jia Ye, S.-H. Gary Chan, Zhenguo Li

Keywords Paper

Chong Chen, Min Zhang, Weizhi Ma and
Yiqun Liu, Shaoping Ma

Keywords Paper

Kibeom Kim, Min Whoo Lee, Yoonsung Kim and
JeHwan Ryu, Minsu Lee, Byoung-Tak Zhang

Keywords Paper

Xiangpeng Wei, Heng Yu, Yue Hu and
Rongxiang Weng, Luxi Xing, Weihua Luo

Keywords Paper

Keywords Paper

Jiaqi Guo, Qian Liu, Jian-Guang Lou and
Zhenwen Li, Xueqing Liu, Tao Xie, Ting Liu

Keywords Paper

Emanuele Bugliarello, Sabrina J. Mielke, Antonios Anastasopoulos and
Ryan Cotterell, Naoaki Okazaki

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Ilias Chalkidis, Manos Fergadiotis, Sotiris Kotitsas and
Prodromos Malakasiotis, Nikolaos Aletras, Ion Androutsopoulos

Keywords Paper

Hao Zhu, Yang Yuan, Guosheng Hu and
Xiang Wu, Neil Robertson

Keywords Paper

Keywords Paper

Keywords Paper

Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat and
Sashank Jakkam Reddi, Sanjiv Kumar

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Yining Hong, Qing Li, Daniel Ciao and
Siyuan Huang, Song-Chun Zhu

Keywords Paper

Keywords Paper

Hiroki Furuta, Tatsuya Matsushima, Tadashi Kozuno and
Yutaka Matsuo, Sergey Levine, Ofir Nachum, Shixiang Gu

Keywords Paper

Zeqi Tan, Yongliang Shen, Shuai Zhang and
Weiming Lu, Yueting Zhuang

Keywords Paper

Keywords Paper

Kaidi Xu, Huan Zhang, Shiqi Wang and
Yihan Wang, Suman Jana, Xue Lin, Cho-Jui Hsieh

Keywords Paper

Keywords Paper

Itay Hubara, Brian Chmiel, Moshe Island and
Ron Banner, Joseph Naor, Daniel Soudry

Keywords Paper

Yuhang Li, Ruihao Gong, Xu Tan and
Yang Yang, Peng Hu, Qi Zhang, fengwei yu, Wei Wang, Shi Gu

Keywords Paper

Keywords Paper

Keywords Paper

Vikash Balasubramanian, Ivan Kobyzev, Hareesh Bahuleyan and
Ilya Shapiro, Olga Vechtomova

Keywords Paper