Abstract:
The success of CNNs (Convolutional Neural Networks) is mainly attributed to the (translation) invariance and local pattern matching effect of convolution kernels. Accordingly, generalizing discrete convolution operation with such two properties (invariance and local pattern matching) to point cloud domain is enlightening for point cloud learning. Inspired by this, we propose an adaptive GMM (Gaussian Mixture Model) convolution (AGMMConv) operation for point cloud learning. Considering the irregularity of point clouds, we propose to represent the kernel points with a GMM, where the mean vectors denote coordinates of kernel points and the covariance matrices determine the shape of each kernel point. Meanwhile, the GMM is a distribution representation of the local geometric surface learned from the local observation, which makes the kernel adaptive to local geometric structures. The proposed convolution is intrinsically invariant to permutation and translation. Besides, potential rotation invariance can be induced from the probability representation, which is an important prior for 3D objects recognition. In convolution, a series of shared weights are associated with each GMM kernel point to match local patterns of point clouds, which allows us to learn rich features with various learnable templates by analogy to the classical image convolution. Experiments on various datasets including object-level and scene-level tasks demonstrate the effectiveness and robustness of the proposed method. Code is available at https://github.com/yangfei1223/AGMMConv.
