Few-Shot Class-Incremental Learning via Relation Knowledge Distillation

02/02/2021

Few-Shot Class-Incremental Learning via Relation Knowledge Distillation

Songlin Dong, Xiaopeng Hong, Xiaoyu Tao, Xinyuan Chang, Xing Wei, Yihong Gong

Keywords:

Abstract Paper Similar Papers

Abstract: In this paper, we focus on the challenging few-shot class incremental learning (FSCIL) problem, which requires to transfer knowledge from old tasks to new ones and solves catastrophic forgetting. We propose the exemplar relation distillation incremental learning framework to balance the tasks of old-knowledge preserving and new-knowledge adaptation. First, we construct an exemplar relation graph to represent the knowledge learned by the original network and update gradually for new tasks learning. Then an exemplar relation loss function for discovering the relation knowledge between different classes is introduced to learn and transfer the structural information in relation graph. A large number of experiments demonstrate that relation knowledge does exist in the exemplars and our approach outperforms other state-of-the-art class-incremental learning methods on the CIFAR100, miniImageNet, and CUB200 datasets.

The video of this talk cannot be embedded. You can watch it here:

https://slideslive.com/38948324

(Link will open in new window)

0

0

0

0

Share

This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment

no comments yet

Similar Papers

06/12/2021

A Mathematical Framework for Quantifying Transferability in Multi-source Transfer Learning

Xinyi Tong, Xiangxiang Xu, Shao-Lun Huang, Lizhong Zheng

Keywords Paper

theory, deep learning, machine learning, vision, transfer learning

2

1

0

0

13:27

06/12/2020

Mitigating Forgetting in Online Continual Learning via Instance-Aware Parameterization

Hung-Jen Chen, An-Chieh Cheng, Da-Cheng Juan and
Wei Wei, Min Sun

Keywords Paper

0

0

0

0

3:23

26/04/2020

Automated Relational Meta-learning

Huaxiu Yao, Xian Wu, Zhiqiang Tao and
Yaliang Li, Bolin Ding, Ruirui Li, Zhenhui Li

Keywords Paper

meta-learning, task heterogeneity, meta-knowledge graph

1

1

0

0

5:13

06/12/2021

Simple Stochastic and Online Gradient Descent Algorithms for Pairwise Learning

ZHENHUAN YANG, Yunwen Lei, Puyu Wang and
Tianbao Yang, Yiming Ying

Keywords Paper

optimization, machine learning, privacy

0

0

0

0

14:40

26/04/2020

Progressive learning and disentanglement of hierarchical representations

Zhiyuan Li, Jaideep Vitthal Murkute, Prashnna Kumar Gyawali, Linwei Wang

Keywords Paper

generative model, disentanglement, progressive learning, VAE

0

0

0

0

5:06

26/04/2020

Gradients as Features for Deep Representation Learning

Fangzhou Mu, Yingyu Liang, Yin Li

Keywords Paper

representation learning, gradient features, deep learning

0

0

0

0

5:07

06/12/2021

Learning to Learn Dense Gaussian Processes for Few-Shot Learning

Ze Wang, Zichen Miao, Xiantong Zhen, Qiang Qiu

Keywords Paper

deep learning, optimization, generative model, meta learning, kernel methods, few shot learning

0

0

0

0

5:21

18/07/2021

A Theory of Label Propagation for Subpopulation Shift

Tianle Cai, Ruiqi Gao, Jason Lee, Qi Lei

Keywords Paper

Theory, Statistical Learning Theory

0

1

0

0

5:08

30/11/2020

Class-incremental Learning with Rectified Feature-Graph Preservation

Cheng-Hsun Lei, Yi-Hsin Chen, Wen-Hsiao Peng, Wei-Chen Chiu

Keywords Paper

0

0

0

0

9:15

06/12/2020

Online Structured Meta-learning

Huaxiu Yao, Yingbo Zhou, Mehrdad Mahdavi and
Zhenhui (Jessie) Li, Richard Socher, Caiming Xiong

Keywords Paper

1

1

0

0

3:28

18/11/2020

CCA-flow: Deep multi-view subspace learning with inverse autoregressive flow

Jia He, Feiyang Pan, Fuzhen Zhuang, Qing He

Keywords Paper

0

0

0

0

11:33

23/08/2020

Neural subgraph isomorphism counting

Xin Liu, Haojie Pan, Mutian He and
Yangqiu Song, Xin Jiang, Lifeng Shang

Keywords Paper

neural network, dynamic memory, subgraph isomorphism

0

0

0

0

13:43

02/02/2021

Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation

Xueyi Li, Tianfei Zhou, Jianwu Li and
Yi Zhou, Zhaoxiang Zhang

Keywords Paper

0

0

0

0

14:47

12/07/2020

Few-shot Relation Extraction via Bayesian Meta-learning on Task Graphs

Meng Qu, Tianyu Gao, Louis-Pascal Xhonneux, Jian Tang

Keywords Paper

Applications - Language, Speech and Dialog

0

0

0

0

6:45

06/12/2021

Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach

Qitian Wu, Chenxiao Yang, Junchi Yan

Keywords Paper

deep learning, machine learning, graph learning, representation learning

0

0

0

0

12:51

02/02/2021

DeepCollaboration: Collaborative Generative and Discriminative Models for Class Incremental Learning

Bo Cui, Guyue Hu, Shan Yu

Keywords Paper

0

0

0

0

15:13

06/12/2021

Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis

Qi CHEN, Changjian Shui, Mario Marchand

Keywords Paper

deep learning, meta learning, few shot learning

0

0

0

0

11:45

23/08/2020

An efficient neighborhood-based interaction model for recommendation on heterogeneous graph

Jiarui Jin, Jiarui Qin, Yuchen Fang and
Kounianhua Du, Weinan Zhang, Yong Yu, Zheng Zhang, Alexander J. Smola

Keywords Paper

recommender system, heterogeneous information network, neighborhood-based interaction

0

0

0

0

13:53

12/07/2020

Graph Filtration Learning

Christoph Hofer, Florian Graf, Bastian Rieck and
Marc Niethammer, Roland Kwitt

Keywords Paper

Deep Learning - General

0

0

0

0

11:21

06/12/2020

Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning

Younggyo Seo, Kimin Lee, Ignasi Clavera Gilaberte and
Thanard Kurutach, Jinwoo Shin, Pieter Abbeel

Keywords Paper

0

0

0

0

3:20

30/11/2020

Large-Scale Cross-Domain Few-Shot Learning

Jiechao Guan, Manli Zhang, Zhiwu Lu

Keywords Paper

0

0

0

0

7:26

02/02/2021

Self-Supervised Hypergraph Convolutional Networks for Session-based Recommendation

Xin Xia, Hongzhi Yin, Junliang Yu and
Qinyong Wang, Lizhen Cui, Xiangliang Zhang

Keywords Paper

0

0

0

0

21:04

06/12/2021

Class-Incremental Learning via Dual Augmentation

Fei Zhu, Zhen Cheng, Xu-yao Zhang, Cheng-lin Liu

Keywords Paper

deep learning, continual learning

0

0

0

0

4:23

03/05/2021

Meta-GMVAE: Mixture of Gaussian VAE for Unsupervised Meta-Learning

Dong Bok Lee, Dongchan Min, Seanie Lee, Sung Ju Hwang

Keywords Paper

Unsupervised Learning, Variational Autoencoders, Unsupervised Meta-learning, Meta-Learning

0

0

0

0

13:31

12/07/2020

Unsupervised Transfer Learning for Spatiotemporal Predictive Networks

Zhiyu Yao, Yunbo Wang, Mingsheng Long, Jianmin Wang

Keywords Paper

Sequential, Network, and Time-Series Modeling

0

0

0

0

15:19

06/12/2021

Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning

Jannik Kossen, Neil Band, Clare Lyle and
Aidan Gomez, Thomas Rainforth, Yarin Gal

Keywords Paper

deep learning, transformers

0

0

0

0

9:54

16/11/2020

Adaptive Attentional Network for Few-Shot Knowledge Graph Completion

Jiawei Sheng, Shu Guo, Zhenyu Chen and
Juwei Yue, Lihong Wang, Tingwen Liu, Hongbo Xu

Keywords Paper

few-shot completion, knowledge acquisition, link prediction, adaptive network

0

0

0

0

11:38

06/12/2020

Joint Contrastive Learning with Infinite Possibilities

Qi Cai, Yu Wang, Yingwei Pan and
Ting Yao, Tao Mei

Keywords Paper

0

0

0

0

3:06

22/11/2021

HCV: Hierarchy-Consistency Verification for Incremental Implicitly-Refined Classification

Kai Wang, Xialei Liu, Luis Herranz, Joost van de Weijer

Keywords Paper

continual learning, multi-label classification, hierarchical classification

0

0

0

0

2:54

03/05/2021

Evaluating the Disentanglement of Deep Generative Models through Manifold Topology

Sharon Zhou, Eric Zelikman, Fred Lu and
Andrew Ng, Gunnar E Carlsson, Stefano Ermon

Keywords Paper

generative models, disentanglement, evaluation

0

0

0

0

5:06

19/08/2021

Regularising Knowledge Transfer by Meta Functional Learning

Pan Li, Yanwei Fu, Shaogang Gong

Keywords Paper

Machine Learning, Classification, Transfer, Adaptation, Multi-task Learning, Weakly Supervised Learning

0

0

0

0

13:41

22/11/2021

SemGIF: A Semantics Guided Incremental Few-shot Learning Framework with Generative Replay

S Divakar Bhat, Biplab Banerjee, Subhasis Chaudhuri

Keywords Paper

Incremental few shot learning, Few shot learning, Incremental learning, feature augmentation, cross dataset, heterogenous

0

0

0

0

3:05

03/08/2020

Robust Collective Classification against Structural Attacks

Kai Zhou, Yevgeniy Vorobeychik

Keywords Paper

0

0

0

0

8:42

06/12/2020

Learning to Learn Variational Semantic Memory

Xiantong Zhen, Yingjun Du, Huan Xiong and
Qiang Qiu, Cees Snoek, Ling Shao

Keywords Paper

0

1

1

1

3:24

06/12/2021

Efficient Learning of Discrete-Continuous Computation Graphs

David Friede, Mathias Niepert

Keywords Paper

deep learning, reinforcement learning and planning, graph learning

0

0

0

0

12:31

19/08/2021

Ordering-Based Causal Discovery with Reinforcement Learning

Xiaoqiang Wang, Yali Du, Shengyu Zhu and
Liangjun Ke, Zhitang Chen, Jianye Hao, Jun Wang

Keywords Paper

Machine Learning Applications, Applications of Reinforcement Learning, Bayesian Networks

0

0

0

0

8:44

06/12/2021

NeuroLKH: Combining Deep Learning Model with Lin-Kernighan-Helsgaun Heuristic for Solving the Traveling Salesman Problem

Liang Xin, Wen Song, Zhiguang Cao, Jie Zhang

Keywords Paper

deep learning, self-supervised learning, graph learning

0

0

0

0

8:14

19/08/2021

Deep Descriptive Clustering

Hongjing Zhang, Ian Davidson

Keywords Paper

Machine Learning, Clustering, Explainable/Interpretable Machine Learning, Constraints and Data Mining; Constraints and Machine Learning

0

0

0

0

15:12

06/12/2020

Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networks

David Bieber, Charles Sutton, Hugo Larochelle, Daniel Tarlow

Keywords Paper

0

0

0

0

3:20

12/07/2020

Learning Structured Latent Factors from Dependent Data:A Generative Model Framework from Information-Theoretic Perspective

Ruixiang ZHANG, Katsuhiko Ishiguro, Masanori Koyama

Keywords Paper

Learning Theory

0

0

0

0

14:46