cs.DC cs.AR cs.LG cs.PF
Time Based Roofline for Deep Learning Performance Analysis. Deep learning applications are usually very compute intensive and require a long run time for training and inference. This has been tackled by researchers from both hardware and software sides, and in this paper, we propose a Roofline based approach to performance analysis to facilitate the optimization of these applications. This approach is an extension of the Roofline model widely used in traditional high performance computing applications, and it incorporates both compute bandwidth complexity and run time in its formulae to provide insights into deep learning specific characteristics. We take two sets of representative kernels, 2D convolution and long short term memory, to validate and demonstrate the use of this new approach, and investigate how arithmetic intensity, cache locality, auto tuning, kernel launch overhead, and Tensor Core usage can affect performance. Compared to the common ad hoc approach, this study helps form a more systematic way to analyze code performance and identify optimization opportunities for deep learning applications.