02/02/2021

Efficient On-Chip Learning for Optical Neural Networks Through Power-Aware Sparse Zeroth-Order Optimization

Jiaqi Gu, Chenghao Feng, Zheng Zhao, Zhoufeng Ying, Ray T. Chen, David Z. Pan

Keywords:

Abstract Paper Similar Papers

Abstract: Optical neural networks (ONNs) have demonstrated record-breaking potential in high-performance neuromorphic computing due to their ultra-high execution speed and low energy consumption. However, current learning protocols fail to provide scalable and efficient solutions to photonic circuit optimization in practical applications. In this work, we propose a novel on-chip learning framework to release the full potential of ONNs for power-efficient in situ training. Instead of deploying implementation-costly back-propagation, we directly optimize the device configurations with computation budgets and power constraints. We are the first to model the ONN on-chip learning as a resource-constrained stochastic noisy zeroth-order optimization problem, and propose a novel mixed-training strategy with two-level sparsity and power-aware dynamic pruning to offer a scalable on-chip training solution in practical ONN deployment. Compared with previous methods, we are the first to optimize over 2,500 optical components on chip. We can achieve much better optimization stability, 3.7x-7.6x higher efficiency, and save >90% power under practical device variations and thermal crosstalk.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948640
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 12:07 
 0
 0
 0
 0
 5:09 
 0
 0
 0
 0
 4:53 
 0
 0
 0
 0
 15:30 
 0
 0
 0
 0
 4:12 
 0
 0
 0
 0
 1:01 
 1
 0
 0
 0
 1:01 
 0
 0
 0
 1
 20:37 
 0
 0
 0
 0
 10:27 
 0
 0
 0
 0
 4:36 
 0
 0
 0
 0
 15:17 
 0
 0
 0
 0
 14:12 
 0
 0
 0
 0
 13:50 
 0
 0
 0
 0
 13:17 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 3:12 
 0
 0
 0
 0
 4:07 
 0
 1
 0
 1
 3:19 
 0
 0
 0
 0
 5:15 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 16:43 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 3:32 
 0
 0
 0
 0
 12:53 
 0
 0
 0
 0
 15:00 
 0
 0
 0
 0
 5:22 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 11:56 
 0
 0
 0
 0
 18:00 
 0
 0
 0
 0
 4:14 
 0
 0
 0
 0
 5:08 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 16:57 
 0
 0
 0
 1
 14:48 
 0
 0
 0
 0
 14:24 
 0
 0
 0
 0
 15:18 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 5:02