05/04/2021

sensAI: ConvNets Decomposition via Class Parallelism for Fast Inference on Live Data

Guanhua Wang, Zhuang Liu, Brandon Hsieh, Siyuan Zhuang, Joseph Gonzalez, Trevor Darrell, Ion Stoica

Keywords:

Abstract Paper Similar Papers

Abstract: Convolutional Neural Networks (ConvNets) enable computers to excel on vision learning tasks such as image classification, object detection. Recently, real-time inference on live data is becoming more and more important. From a system perspective, it requires fast inference on each single, incoming data item (e.g. 1 image). Two main-stream distributed model serving paradigms – data parallelism and model parallelism – are not necessarily desirable here, because we cannot further split a single input data piece via data parallelism, and model parallelism introduces huge communication overhead. To achieve live data inference with low latency, we propose sensAI, a novel and generic approach that decouples a CNN model into disconnected subnets, each is responsible for predicting certain class(es). We call this new model distribution paradigm as class parallelism. Experimental results show that, sensAI achieves up to 18x faster inference on single input data item with no or negligible accuracy loss on CIFAR-10, CIFAR-100 and ImageNet-1K datasets.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38952769
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at MLSYS 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 5:23 
 0
 0
 0
 0
 10:42 
 0
 0
 0
 0
 4:44 
 0
 0
 0
 0
 18:27 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 19:59 
 0
 0
 0
 0
 4:55 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 9:54 
 0
 0
 0
 0
 5:10 
 0
 0
 0
 0
 1:03 
 0
 0
 0
 0
 0:53 
 0
 0
 0
 0
 19:55 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 18:05 
 0
 0
 0
 0
 8:50 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 8:06 
 0
 0
 0
 0
 6:36 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 16:06 
 0
 0
 0
 0
 7:30 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 0:59 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 0:58 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 4:01 
 0
 0
 0
 0
 14:59 
 0
 0
 0
 0
 4:12 
 0
 0
 0
 0
 8:30 
 0
 0
 0
 0
 4:24 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 11:18 
 0
 0
 0
 0
 10:12 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 0:57