SDP-Net: Scene Flow Based Real-time Object Detection and Prediction from Sequential 3D Point Clouds

30/11/2020

SDP-Net: Scene Flow Based Real-time Object Detection and Prediction from Sequential 3D Point Clouds

Yi Zhang, Yuwen Ye, Zhiyu Xiang, Jiaqi Gu

Keywords:

Abstract Paper Similar Papers

Abstract: Robust object detection in 3D point clouds faces the challenges caused by sparse range data. Accumulating multi-frame data could densify the 3D point clouds and greatly benefit detection task. However, accurately aligning the point clouds before the detecting process is a difficult task since there may exist moving objects in the scene. In this paper a novel scene flow based multi-frame network named SDP-Net is proposed. It is able to perform multiple tasks such as self-alignment, 3D object detection, prediction and tracking simultaneously. Thanks to the design of scene flow and the scheme of multi-task, our network is capable of working effectively with a simple network backbone. We further improve the annotations on KITTI RAW dataset by supplementing the ground truth. Experimental results show that our approach greatly outperforms the state-of-the-art and can perform multiple tasks in real-time.

The video of this talk cannot be embedded. You can watch it here:

https://accv2020.github.io/miniconf/poster_599.html

(Link will open in new window)

0

0

0

0

Share

This is an embedded video. Talk and the respective paper are published at ACCV 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment

no comments yet

Similar Papers

14/06/2020

PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

Shaoshuai Shi, Chaoxu Guo, Li Jiang and
Zhe Wang, Jianping Shi, Xiaogang Wang, Hongsheng Li

Keywords Paper

3d object detection, point cloud, 3d scene understanding, lidar, autonomous driving, kitti dataset, waymo open dataset

0

0

0

0

1:01

14/06/2020

Associate-3Ddet: Perceptual-to-Conceptual Association for 3D Point Cloud Object Detection

Liang Du, Xiaoqing Ye, Xiao Tan and
Jianfeng Feng, Zhenbo Xu, Errui Ding, Shilei Wen

Keywords Paper

3d object detection, domain adaptation, associative recognition, lidar, point cloud, convolutional neural network, autonomous driving

0

0

0

0

1:01

05/01/2021

Triangle-Net: Towards Robustness in Point Cloud Learning

Chenxi Xiao, Juan Wachs

Keywords Paper

0

0

0

0

4:58

14/06/2020

Neural Point Cloud Rendering via Multi-Plane Projection

Peng Dai, Yinda Zhang, Zhuwen Li and
Shuaicheng Liu, Bing Zeng

Keywords Paper

neural rendering, point cloud, multi-plane representation, novel view synthesis

0

0

0

0

1:00

30/11/2020

Bi-Directional Attention for Joint Instance and Semantic Segmentation in Point Clouds

guangnan wu, Zhiyi Pan, Peng Jiang, Changhe Tu

Keywords Paper

0

0

0

0

9:15

14/06/2020

SESS: Self-Ensembling Semi-Supervised 3D Object Detection

Na Zhao, Tat-Seng Chua, Gim Hee Lee

Keywords Paper

3d object detection, semi-supervised learning, self-ensembling technique, point cloud analysis

0

0

0

0

5:01

14/06/2020

OccuSeg: Occupancy-Aware 3D Instance Segmentation

Lei Han, Tian Zheng, Lan Xu, Lu Fang

Keywords Paper

instance segmentation, multi-task learning, occupancy

0

0

0

0

1:02

02/02/2021

PC-RGNN: Point Cloud Completion and Graph Neural Network for 3D Object Detection

Yanan Zhang, Di Huang, Yunhong Wang

Keywords Paper

0

0

0

0

16:30

14/06/2020

ImVoteNet: Boosting 3D Object Detection in Point Clouds With Image Votes

Charles R. Qi, Xinlei Chen, Or Litany, Leonidas J. Guibas

Keywords Paper

3d object detection, rgb-d, voting, point clouds, multi-modality, fusion, deep learning, object recognition.

0

0

0

0

1:00

14/06/2020

Density-Based Clustering for 3D Object Detection in Point Clouds

Syeda Mariam Ahmed, Chee Meng Chew

Keywords Paper

3d object detection, edge-aware pointnet, instance segmentation, unsupervised clustering, cascaded modules, semantic segmentation, amodal bounding box detection

0

0

0

0

0:51

06/12/2021

Progressive Coordinate Transforms for Monocular 3D Object Detection

Li Wang, Li Zhang, Yi Zhu and
Zhi Zhang, Tong He, Mu Li, Xiangyang Xue

Keywords Paper

vision

0

0

0

0

13:21

22/11/2021

Two Heads are Better than One: Geometric-Latent Attention for Point Cloud Classification and Segmentation

Hanz Cuevas, Antonio-Javier Gallego, Robert B Fisher

Keywords Paper

point cloud, point cloud segmentation, 3D, point cloud classification, shape classification, deep 3D segmentation, deep learning, scene understanding, 3D scene understanding, light network, local agregation, gnn

0

0

0

0

2:57

22/11/2021

Planar Shape Based Registration for Multi-modal Geometry

Muxingzi Li, Florent Lafarge

Keywords Paper

global registration, energy minimization, geometric primitives, point cloud, polygonal mesh

0

0

0

0

3:00

14/06/2020

PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation

Yisheng He, Wei Sun, Haibin Huang and
Jianran Liu, Haoqiang Fan, Jian Sun

Keywords Paper

6d pose estimation, 3d instance segmentation, 3d semantic segmentation, 3d keypoint, 3d scene understanding, vision for robotics, 3d single view, rgbd, 3d computer vision

0

0

0

0

1:01

22/11/2021

Enhancing Local Feature Learning for 3D Point Cloud Processing using Unary-Pairwise Attention

Haoyi Xiu, Xin Liu, Weimin Wang and
Kyoung-Sook Kim, Takayuki Shinohara, Qiong Chang, Masashi Matsuoka

Keywords Paper

3D point clouds, self-attention, 3D point cloud deep learning

0

0

0

0

2:51

05/01/2021

Learning to Generate Dense Point Clouds With Textures on Multiple Categories

Tao Hu, Geng Lin, Zhizhong Han, Matthias Zwicker

Keywords Paper

0

0

0

0

4:57

02/02/2021

Object-Centric Image Generation from Layouts

Tristan Sylvain, Pengchuan Zhang, Yoshua Bengio and
R Devon Hjelm, Shikhar Sharma

Keywords Paper

0

0

0

0

17:44

06/12/2021

Twins: Revisiting the Design of Spatial Attention in Vision Transformers

Xiangxiang Chu, Zhi Tian, Yuqing Wang and
Bo Zhang, Haibing Ren, Xiaolin Wei, Huaxia Xia, Chunhua Shen

Keywords Paper

deep learning, machine learning, transformers, vision

0

0

0

0

5:29

06/12/2021

Shape from Blur: Recovering Textured 3D Shape and Motion of Fast Moving Objects

Denys Rozumnyi, Martin R. Oswald, Vittorio Ferrari, Marc Pollefeys

Keywords Paper

optimization

0

0

0

0

10:44

14/06/2020

PointRend: Image Segmentation As Rendering

Alexander Kirillov, Yuxin Wu, Kaiming He, Ross Girshick

Keywords Paper

instance segmentation, semantic segmentation, high-resolution, implicit function, rendering

0

0

0

0

4:59

06/12/2020

Fine-Grained Dynamic Head for Object Detection

Lin Song, Yanwei Li, Zhengkai Jiang and
Zeming Li, Hongbin Sun, Jian Sun, Nanning Zheng

Keywords Paper

Applications -> Computational Biology and Bioinformatics; Applications -> Health; Applications -> Time Series Analysis; Neurosc, Neuroscience and Cognitive Science -> Brain Imaging

0

0

0

0

3:19

30/11/2020

3D Guided Weakly Supervised Semantic Segmentation

Weixuan Sun, Jing Zhang, Nick Barnes

Keywords Paper

0

0

0

0

9:20

30/11/2020

IAFA: Instance-Aware Feature Aggregation for 3D Object Detection from a Single Image

Dingfu Zhou, Xibin Song, Yuchao Dai and
Junbo Yin, Feixiang Lu, Miao Liao, Jin Fang, Liangjun Zhang

Keywords Paper

0

0

0

0

6:20

14/06/2020

BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation

Hao Chen, Kunyang Sun, Zhi Tian and
Chunhua Shen, Yongming Huang, Youliang Yan

Keywords Paper

instance segmentation, fully-convolutional, object detection, real-time

0

0

0

0

4:39

06/12/2021

Sparse Steerable Convolutions: An Efficient Learning of SE(3)-Equivariant Features for Estimation and Tracking of Object Poses in 3D Space

Jiehong Lin, Hongyang Li, Ke Chen and
Jiangbo Lu, Kui Jia

Keywords Paper

vision

0

0

0

0

12:29

14/06/2020

Weakly Supervised Semantic Point Cloud Segmentation: Towards 10× Fewer Labels

Xun Xu, Gim Hee Lee

Keywords Paper

3d point cloud, semantic segmentation, weakly supervised learning, semi-supervised learning

0

0

0

0

1:01

19/08/2021

Sequential 3D Human Pose Estimation Using Adaptive Point Cloud Sampling Strategy

Zihao Zhang, Lei Hu, Xiaoming Deng, Shihong Xia

Keywords Paper

Computer Vision, 2D and 3D Computer Vision, Motion and Tracking, Deep Learning

0

0

0

0

13:34

07/09/2020

Attribute-Guided Image Generation from Layout

Ke Ma, Bo Zhao, Leonid Sigal

Keywords Paper

conditional image generation, GAN

0

0

0

0

9:41

14/06/2020

Connect-and-Slice: An Hybrid Approach for Reconstructing 3D Objects

Hao Fang, Florent Lafarge

Keywords Paper

surface reconstruction, surface approximation, polygonal surface mesh, plane arrangement, point cloud

0

0

0

0

1:01

14/06/2020

Shape Reconstruction by Learning Differentiable Surface Representations

Jan Bednařík, Shaifali Parashar, Erhan Gündoğdu and
Mathieu Salzmann, Pascal Fua

Keywords Paper

differentiable geometry, surface reconstruction, surface representation, scaled isometry, surface normals, surface curvature, point cloud autoencoding, single view reconstruction, shape completion, metric tensor

0

0

0

0

1:01

14/06/2020

What You See is What You Get: Exploiting Visibility for 3D Object Detection

Peiyun Hu, Jason Ziglar, David Held, Deva Ramanan

Keywords Paper

freespace reasoning, 3d object detection, lidar processing, autonomous driving

0

0

0

0

5:01

14/06/2020

ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning

Weiwei Sun, Wei Jiang, Eduard Trulls and
Andrea Tagliasacchi, Kwang Moo Yi

Keywords Paper

point clouds, attention, normalization, iterative reweighted least squares, permutation-equivariance, wide-baseline stereo, point cloud classification, nerual line fitting, pointnet, context normalization

0

0

0

0

1:01

14/06/2020

ASLFeat: Learning Local Features of Accurate Shape and Localization

Zixin Luo, Lei Zhou, Xuyang Bai and
Hongkai Chen, Jiahui Zhang, Yao Yao, Shiwei Li, Tian Fang, Long Quan

Keywords Paper

image matching, local feature keypoints, local feature descriptors, deep learning

0

0

0

0

1:01

06/12/2021

Knowledge-inspired 3D Scene Graph Prediction in Point Cloud

Shoulong Zhang, shuai li, Aimin Hao, Hong Qin

Keywords Paper

deep learning, graph learning

0

0

0

0

11:20

02/02/2021

Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion

Xu Yan, Jiantao Gao, Jie Li and
Ruimao Zhang, Zhen Li, Rui Huang, Shuguang Cui

Keywords Paper

0

0

0

0

14:24

14/06/2020

Hierarchical Scene Coordinate Classification and Regression for Visual Localization

Xiaotian Li, Shuzhe Wang, Yi Zhao and
Jakob Verbeek, Juho Kannala

Keywords Paper

visual localization, camera relocalization, scene coordinate regression

0

0

0

0

1:01

02/02/2021

Exploiting Relationship for Complex-scene Image Generation

Tianyu Hua, Hongdong Zheng, Yalong Bai and
Wei Zhang, Xiao-Ping Zhang, Tao Mei

Keywords Paper

0

0

0

0

15:01

02/02/2021

DenserNet: Weakly Supervised Visual Localization Using Multi-Scale Feature Aggregation

Dongfang Liu, Yiming Cui, Liqi Yan and
Christos Mousas, Baijian Yang, Yingjie Chen

Keywords Paper

0

0

0

0

16:15

03/05/2021

Learning Long-term Visual Dynamics with Region Proposal Interaction Networks

Haozhi Qi, Xiaolong Wang, Deepak Pathak and
Yi Ma, Jitendra Malik

Keywords Paper

dynamics prediction, physical reasoning, interaction networks

0

0

0

0

5:10

07/09/2020

Advancing weakly supervised cross-domain alignment with optimal transport

Siyang Yuan, Ke Bai, Liqun Chen and
Yizhe Zhang, Chenyang Tao, Chunyuan Li, Guoyin Wang, Ricardo Henao, Lawrence Carin Duke

Keywords Paper

Optimal Transport, Cross Domain Alignment

0

0

0

0

10:04