Abstract:
Meta-learning approaches have shown great success in solving challenging knowledge transfer and fast adaptation problems with few samples in vision and language domains. However, few studies discuss the practice of meta-learning for large-scale industrial applications, e.g., representation learning for e-commerce platform users. Although e-commerce companies have spent many efforts on learning accurate and expressive representations to provide a better user experience, we argue that such efforts cannot be stopped at this step. In addition to learning a strong profile of user behaviors, the challenging question about how to effectively transfer the learned representation and quickly adapt the learning process to the subsequent learning tasks or applications is raised simultaneously.This paper introduces the contributions that we made to address these challenges from three aspects. 1) Meta-learning model: In the context of representation learning with e-commerce user behavior data, we propose a meta-learning framework called the Meta-Profile Network, which extends the ideas of matching network and relation network for knowledge transfer and fast adaptation; 2) Encoding strategy: To keep high fidelity of large-scale long-term sequential behavior data, we propose a time-heatmap encoding strategy that allows the model to encode data effectively; 3) Deep network architecture: A multi-modal model combined with multi-task learning architecture is utilized to address the cross-domain knowledge learning and insufficient label problems. Moreover, we argue that an industrial model should not only have good performance in terms of accuracy, but also have better robustness and uncertainty performance under extreme conditions. We evaluate the performance of our model with extensive control experiments in various extreme scenarios, i.e. out-of-distribution detection, data insufficiency and class imbalance scenarios. The Meta-Profile Network shows significant improvement in the model performance when compared to baseline models.
