Training time problem

[lahmuller]

It seems to me that training with the sparse model(model=model.to_sparse()) costs way more time than dense model. I have tried the code in 2020 and 2021, but the results are same: sparse costs more time. Does this has anything to do with my system(win10)？ Hope the answer. Thank you.

[jiangyuang]

@lahmuller Thanks for the question. In our documentation (also see our Arxiv paper Section C.3), we stated

extension: the extension.cpp c++ file extends the current PyTorch implementation with sparse kernels (the installed module is called sparse_conv2d). However, please note that we only extend PyTorch's slow, cpu version of conv2d forward/backward with no groups and dilation = 1 (see PyTorch's c++ code here). In other words, we do not use acceleration packages such as MKL (which are not available on Raspberry Pis on which our paper experimented). Do not compare the speed of our implementation with the acceleration packages.

So please do not compare with the performance of the code that uses any acceleration package such as MKL. Note that PyTorch automatically use such acceleration packages unless configured specifically at installation. Also, the sparse model is faster only when its density is small, e.g. try < 10% density.

[lahmuller]

@lahmuller Thanks for the question. In our documentation (also see our Arxiv paper Section C.3), we stated

extension: the extension.cpp c++ file extends the current PyTorch implementation with sparse kernels (the installed module is called sparse_conv2d). However, please note that we only extend PyTorch's slow, cpu version of conv2d forward/backward with no groups and dilation = 1 (see PyTorch's c++ code here). In other words, we do not use acceleration packages such as MKL (which are not available on Raspberry Pis on which our paper experimented). Do not compare the speed of our implementation with the acceleration packages.

So please do not compare with the performance of the code that uses any acceleration package such as MKL. Note that PyTorch automatically use such acceleration packages unless configured specifically at installation. Also, the sparse model is faster only when its density is small, e.g. try < 10% density.

I thought pytorch will also accelerate sparse model as well as dense model since they are in the same env. Maybe I should create a new env without acceleration package to try this out. Thank you for your specification.

[Karry-Xu]

Hi,I tried to use your sparse matrix kernel to speed up, but it didn't work. You mentioned that Python will automatically use MKL to speed up. Would you like to make a comparative experiment to show the acceleration performance?

[jiangyuang]

@Karry-Xu We experimented the models mentioned in our paper on Raspberry Pi (version 4), where packaged such as MKL are automatically not used since Pi's use Arm-based CPUs.

In the table below, we show the time for one training iteration with 20 mini-batch size, both in dense form, and the corresponding time in sparse form where every layer only has one weight. The time in sparse form means that you can save at most 40% to 50% time (when your model is small enough using sparse form).

	Dense form time	Sparse form time with each parameterized layer having one weight
Conv2	2.2486449218005875	1.2492789765
Conv4	3.724286518478766	1.598415535595268
VGG-11	31.514276721399803	17.36990105989628
Resnet-18	25.3356386726	14.560097051459854

[Karry-Xu]

@jiangyuang Thanks for your reply