Skip to content

[mobile_perf][recipe] Add ChannelsLast recommendation #1042

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 2, 2020
Merged

[mobile_perf][recipe] Add ChannelsLast recommendation #1042

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9cc3ce8
[mobile_perf][recipe] Add ChannelsLast recommendation
IvanKobzarev Jun 26, 2020
e6ac673
Merge branch 'release/1.6' into mobile_perf_channels_last
Jul 2, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions recipes_source/mobile_perf.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ Code your model:
self.dequant = torch.quantization.DeQuantStub()

def forward(self, x):
x.contiguous(memory_format=torch.channels_last)
x = self.quant(x)
x = self.conv(x)
x = self.bn(x)
Expand Down Expand Up @@ -134,9 +135,24 @@ Next we call ``optimize_for_mobile`` and save model on the disk.
torchscript_model_optimized = optimize_for_mobile(torchscript_model)
torch.jit.save(torchscript_model_optimized, "model.pt")

4. Prefer Using Channels Last Tensor memory format
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

4. Android. Reusing tensors for forward.
#############################################################
Channels Last(NHWC) memory format was introduced in PyTorch 1.4.0. It is supported only for four-dimensional tensors. This memory format gives a better memory locality for most operators, especially convolution. Our measurements showed a 3x speedup of MobileNetV2 model compared with the default Channels First(NCHW) format.

At the moment of writing this recipe, PyTorch Android java API does not support using inputs in Channels Last memory format. But it can be used on the TorchScript model level, by adding the conversion to it for model inputs.

.. code-block:: python

def forward(self, x):
x.contiguous(memory_format=torch.channels_last)
...


This conversion is zero cost if your input is already in Channels Last memory format. After it, all operators will work preserving ChannelsLast memory format.

5. Android. Reusing tensors for forward
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This recipe is Android only.
Memory is a critical resource for android performance, especially on old devices.
Expand Down