Skip to content

Commit 2a10083

Browse files
brianjobrianjo
authored
Revert "Update lite interpreter tutorial to beta (#1549)" (#1569)
This reverts commit a702ca0.
1 parent a702ca0 commit 2a10083

File tree

2 files changed

+221
-198
lines changed

2 files changed

+221
-198
lines changed

‎prototype_source/lite_interpreter.rst‎

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
(Prototype) Introduce lite interpreter workflow in Android and iOS
2+
==================================================================
3+
4+
**Author**: `Chen Lai <https://github.com/cccclai>`_, `Martin Yuan <https://github.com/iseeyuan>`_
5+
6+
Introduction
7+
------------
8+
9+
This tutorial introduces the steps to use lite interpreter on iOS and Android. We'll be using the ImageSegmentation demo app as an example. Since lite interpreter is currently in the prototype stage, a custom pytorch binary from source is required.
10+
11+
12+
Android
13+
-------------------
14+
Get ImageSegmentation demo app in Android: https://github.com/pytorch/android-demo-app/tree/master/ImageSegmentation
15+
16+
1. **Prepare model**: Prepare the lite interpreter version of model by run the script below to generate the scripted model `deeplabv3_scripted.pt` and `deeplabv3_scripted.ptl`
17+
18+
.. code:: python
19+
20+
import torch
21+
22+
model = torch.hub.load('pytorch/vision:v0.7.0', 'deeplabv3_resnet50', pretrained=True)
23+
model.eval()
24+
25+
scripted_module = torch.jit.script(model)
26+
# Export full jit version model (not compatible lite interpreter), leave it here for comparison
27+
scripted_module.save("deeplabv3_scripted.pt")
28+
# Export lite interpreter version model (compatible with lite interpreter)
29+
scripted_module._save_for_lite_interpreter("deeplabv3_scripted.ptl")
30+
31+
2. **Build libtorch lite for android**: Build libtorch for android for all 4 android abis (``armeabi-v7a``, ``arm64-v8a``, ``x86``, ``x86_64``) ``BUILD_LITE_INTERPRETER=1 ./scripts/build_pytorch_android.sh``. For example, if it will be tested on Pixel 4 emulator with ``x86``, use cmd ``BUILD_LITE_INTERPRETER=1 ./scripts/build_pytorch_android.sh x86`` to specify abi to save built time. After the build finish, it will show the library path:
32+
33+
34+
.. code-block:: bash
35+
36+
BUILD SUCCESSFUL in 55s
37+
134 actionable tasks: 22 executed, 112 up-to-date
38+
+ find /Users/chenlai/pytorch/android -type f -name '*aar'
39+
+ xargs ls -lah
40+
-rw-r--r-- 1 chenlai staff 13M Feb 11 11:48 /Users/chenlai/pytorch/android/pytorch_android/build/outputs/aar/pytorch_android-release.aar
41+
-rw-r--r-- 1 chenlai staff 36K Feb 9 16:45 /Users/chenlai/pytorch/android/pytorch_android_torchvision/build/outputs/aar/pytorch_android_torchvision-release.aar
42+
43+
3. **Use the PyTorch Android libraries built from source in the ImageSegmentation app**: Create a folder `libs` in the path, the path from repository root will be `ImageSegmentation/app/libs`. Copy `pytorch_android-release` to the path ``ImageSegmentation/app/libs/pytorch_android-release.aar``. Copy `pytorch_android_torchvision` (downloaded from `Pytorch Android Torchvision Nightly <https://oss.sonatype.org/#nexus-search;quick~torchvision_android/>`_) to the path ``ImageSegmentation/app/libs/pytorch_android_torchvision.aar``. Update the `dependencies` part of ``ImageSegmentation/app/build.gradle`` to
44+
45+
.. code:: gradle
46+
47+
dependencies {
48+
implementation 'androidx.appcompat:appcompat:1.2.0'
49+
implementation 'androidx.constraintlayout:constraintlayout:2.0.2'
50+
testImplementation 'junit:junit:4.12'
51+
androidTestImplementation 'androidx.test.ext:junit:1.1.2'
52+
androidTestImplementation 'androidx.test.espresso:espresso-core:3.3.0'
53+
54+
55+
implementation(name:'pytorch_android-release', ext:'aar')
56+
implementation(name:'pytorch_android_torchvision', ext:'aar')
57+
58+
implementation 'com.android.support:appcompat-v7:28.0.0'
59+
implementation 'com.facebook.fbjni:fbjni-java-only:0.0.3'
60+
}
61+
62+
Update `all projects` part in ``ImageSegmentation/build.gradle`` to
63+
64+
65+
.. code:: gradle
66+
67+
allprojects {
68+
repositories {
69+
google()
70+
jcenter()
71+
flatDir {
72+
dirs 'libs'
73+
}
74+
}
75+
}
76+
77+
4. **Update model loader api**: Update ``ImageSegmentation/app/src/main/java/org/pytorch/imagesegmentation/MainActivity.java`` by
78+
79+
4.1 Add new import: `import org.pytorch.LiteModuleLoader`
80+
81+
4.2 Replace the way to load pytorch lite model
82+
83+
.. code:: java
84+
85+
// mModule = Module.load(MainActivity.assetFilePath(getApplicationContext(), "deeplabv3_scripted.pt"));
86+
mModule = LiteModuleLoader.load(MainActivity.assetFilePath(getApplicationContext(), "deeplabv3_scripted.ptl"));
87+
88+
5. **Test app**: Build and run the `ImageSegmentation` app in Android Studio
89+
90+
iOS
91+
-------------------
92+
Get ImageSegmentation demo app in iOS: https://github.com/pytorch/ios-demo-app/tree/master/ImageSegmentation
93+
94+
1. **Prepare model**: Same as Android.
95+
96+
2. **Build libtorch lite for iOS**:
97+
98+
.. code-block:: bash
99+
100+
BUILD_PYTORCH_MOBILE=1 IOS_PLATFORM=SIMULATOR BUILD_LITE_INTERPRETER=1 ./scripts/build_ios.sh
101+
102+
103+
3. **Remove Cocoapods from the project** (this step is only needed if you ran `pod install`):
104+
105+
.. code-block:: bash
106+
107+
pod deintegrate
108+
109+
4. **Link ImageSegmentation demo app with the custom built library**:
110+
Open your project in XCode, go to your project Target’s **Build Phases - Link Binaries With Libraries**, click the **+** sign and add all the library files located in `build_ios/install/lib`. Navigate to the project **Build Settings**, set the value **Header Search Paths** to `build_ios/install/include` and **Library Search Paths** to `build_ios/install/lib`.
111+
In the build settings, search for **other linker flags**. Add a custom linker flag below
112+
```
113+
-all_load
114+
```
115+
Finally, disable bitcode for your target by selecting the Build Settings, searching for Enable Bitcode, and set the value to **No**.
116+
117+
5. **Update library and api**
118+
119+
5.1 Update ``TorchModule.mm``: To use the custom built libraries the project, replace `#import <LibTorch/LibTorch.h>` (in ``TorchModule.mm``) which is needed when using LibTorch via Cocoapods with the code below:
120+
121+
.. code-block:: swift
122+
123+
//#import <LibTorch/LibTorch.h>
124+
#include "ATen/ATen.h"
125+
#include "caffe2/core/timer.h"
126+
#include "caffe2/utils/string_utils.h"
127+
#include "torch/csrc/autograd/grad_mode.h"
128+
#include "torch/script.h"
129+
#include <torch/csrc/jit/mobile/function.h>
130+
#include <torch/csrc/jit/mobile/import.h>
131+
#include <torch/csrc/jit/mobile/interpreter.h>
132+
#include <torch/csrc/jit/mobile/module.h>
133+
#include <torch/csrc/jit/mobile/observer.h>
134+
135+
.. code-block:: swift
136+
137+
@implementation TorchModule {
138+
@protected
139+
// torch::jit::script::Module _impl;
140+
torch::jit::mobile::Module _impl;
141+
}
142+
143+
- (nullable instancetype)initWithFileAtPath:(NSString*)filePath {
144+
self = [super init];
145+
if (self) {
146+
try {
147+
_impl = torch::jit::_load_for_mobile(filePath.UTF8String);
148+
// _impl = torch::jit::load(filePath.UTF8String);
149+
// _impl.eval();
150+
} catch (const std::exception& exception) {
151+
NSLog(@"%s", exception.what());
152+
return nil;
153+
}
154+
}
155+
return self;
156+
}
157+
158+
159+
5.2 Update ``ViewController.swift``
160+
161+
.. code-block:: swift
162+
163+
// if let filePath = Bundle.main.path(forResource:
164+
// "deeplabv3_scripted", ofType: "pt"),
165+
// let module = TorchModule(fileAtPath: filePath) {
166+
// return module
167+
// } else {
168+
// fatalError("Can't find the model file!")
169+
// }
170+
if let filePath = Bundle.main.path(forResource:
171+
"deeplabv3_scripted", ofType: "ptl"),
172+
let module = TorchModule(fileAtPath: filePath) {
173+
return module
174+
} else {
175+
fatalError("Can't find the model file!")
176+
}
177+
178+
6. Build and test the app in Xcode.
179+
180+
How to use lite interpreter + custom build
181+
------------------------------------------
182+
1. To dump the operators in your model, say `deeplabv3_scripted`, run the following lines of Python code:
183+
184+
.. code-block:: python
185+
186+
# Dump list of operators used by deeplabv3_scripted:
187+
import torch, yaml
188+
model = torch.jit.load('deeplabv3_scripted.ptl')
189+
ops = torch.jit.export_opnames(model)
190+
with open('deeplabv3_scripted.yaml', 'w') as output:
191+
yaml.dump(ops, output)
192+
193+
In the snippet above, you first need to load the ScriptModule. Then, use export_opnames to return a list of operator names of the ScriptModule and its submodules. Lastly, save the result in a yaml file. The yaml file can be generated for any PyTorch 1.4.0 or above version. You can do that by checking the value of `torch.__version__`.
194+
195+
2. To run the build script locally with the prepared yaml list of operators, pass in the yaml file generate from the last step into the environment variable SELECTED_OP_LIST. Also in the arguments, specify BUILD_PYTORCH_MOBILE=1 as well as the platform/architechture type.
196+
197+
**iOS**: Take the simulator build for example, the command should be:
198+
199+
.. code-block:: bash
200+
201+
SELECTED_OP_LIST=deeplabv3_scripted.yaml BUILD_PYTORCH_MOBILE=1 IOS_PLATFORM=SIMULATOR BUILD_LITE_INTERPRETER=1 ./scripts/build_ios.sh
202+
203+
**Android**: Take the x86 build for example, the command should be:
204+
205+
.. code-block:: bash
206+
207+
SELECTED_OP_LIST=deeplabv3_scripted.yaml BUILD_LITE_INTERPRETER=1 ./scripts/build_pytorch_android.sh x86
208+
209+
210+
Conclusion
211+
----------
212+
213+
In this tutorial, we demonstrated how to use lite interpreter in Android and iOS app. We walked through an Image Segmentation example to show how to dump the model, build torch library from source and use the new api to run model. Please be aware of that lite interpreter is still under development, more library size reduction will be introduced in the future. APIs are subject to change in the future versions.
214+
215+
Thanks for reading! As always, we welcome any feedback, so please create an issue `here <https://github.com/pytorch/pytorch/issues>`_ if you have any.
216+
217+
Learn More
218+
----------
219+
220+
- To learn more about PyTorch Mobile, please refer to `PyTorch Mobile Home Page <https://pytorch.org/mobile/home/>`_
221+
- To learn more about Image Segmentation, please refer to the `Image Segmentation DeepLabV3 on Android Recipe <https://pytorch.org/tutorials/beginner/deeplabv3_on_android.html>`_

0 commit comments

Comments
 (0)