Skip to content

Commit dd55883

Browse files
terfendailazhogovdkurtalvoronl-bat
authored
Backport of updates from release 2021.3 branch (opencv#72)
* Azure CI: Add "ref: releases/2021/3" * Fix Klocwork issues in Java * Updated arm plugin build script (opencv#65) * Fix some Klocwork issues * Update README.md (opencv#76) * Fixed Convolution fusion (opencv#84) * Remove MO deps from mo_pytorch module (opencv#93) Co-authored-by: azhogov <[email protected]> Co-authored-by: Dmitry Kurtaev <[email protected]> Co-authored-by: Aleksandr Voron <[email protected]> Co-authored-by: Liubov Batanina <[email protected]>
1 parent 2fbbd7a commit dd55883

File tree

11 files changed

+79
-42
lines changed

11 files changed

+79
-42
lines changed

‎.ci/azure/linux.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,7 @@ jobs:
121121
- script: |
122122
python3 -m pip install -U setuptools==51.0.0
123123
python3 -m pip install -r requirements.txt
124+
python3 -m pip install -r $(OPENVINO_REPO_DIR)/model-optimizer/requirements_onnx.txt
124125
workingDirectory: $(REPO_DIR)/modules/mo_pytorch/test
125126
displayName: 'Install PyTorch tests dependencies'
126127

‎modules/arm_plugin/README.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ docker container run --rm -ti -v $PWD/build:/armcpu_plugin arm-plugin
4343
or export only the archive with artifacts:
4444
```
4545
docker container run --rm -ti --tmpfs /armcpu_plugin:rw -v $PWD:/remote \
46-
arm-plugin sh -c "/armplg_build.sh && cp ./OV_ARM_package.tar.gz /remote"
46+
arm-plugin sh -c "sh /armplg_build.sh && cp ./OV_ARM_package.tar.gz /remote"
4747
```
4848
> **NOTE**: There are a few environment variables that control `/armplg_build.sh` script execution.
4949
>
@@ -189,16 +189,16 @@ On the output video you should see people enclosed in red rectangles:
189189
![](docs/img/object_detection_demo_yolo.gif)
190190

191191
### OpenVINO™ samples
192-
You could verify the plugin by running [OpenVINO™ samples]. You can find C++ samples in `deployment_tools/inference_engine/bin` directory (if you build the plugin using approach #1) or `openvino/bin/armv7l/Release` directory (if you build the plugin using approach #2 or #3). The following procedure assumes the approach #1 is used.
193-
OpenVINO™ samples require OpenCV libraries. If you build the plugin using approach #1 all needed OpenCV libraries are already placed in `build\lib` directory. If you build the plugin using approach #2 or #3 you need to install OpenCV or [build it from source].
192+
You could verify the plugin by running [OpenVINO™ samples]. You can find C++ samples in `deployment_tools/inference_engine/bin/<platform_type>` directory (if you build the plugin using approach #1) or `openvino/bin/<platform_type>/Release` directory (if you build the plugin using approach #2 or #3). The following procedure assumes the approach #1 is used.
193+
OpenVINO™ samples require OpenCV libraries. If you build the plugin using approach #1 all needed OpenCV libraries are already placed in `opencv\lib` directory. If you build the plugin using approach #2 or #3 you need to install OpenCV or [build it from source].
194194
Let's try to run [Object Detection for SSD sample].
195195
#### Model preparation
196196
1. Prepare model `vehicle-license-plate-detection-barrier-0123` using Model Preparation precedure described in Open Model Zoo demos section.
197197
#### Model inference on ARM
198198
1. Copy OpenVINO™ and ARM plugin artefacts to ARM platform. If you build the plugin using approach #1, all artefacts are packed into `OV_ARM_package.tar.gz`.
199-
2. Go to `deployment_tools/inference_engine/bin` directory:
199+
2. Go to Inference Engine bin directory:
200200
```
201-
cd deployment_tools/inference_engine/bin
201+
cd deployment_tools/inference_engine/bin/<platform_type>
202202
```
203203
3. Download a vehicle image, for instance, [this image]:
204204
```
@@ -208,7 +208,7 @@ wget https://raw.githubusercontent.com/openvinotoolkit/openvino/master/scripts/d
208208

209209
5. Run object detection sample on ARM platform:
210210
```
211-
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD/lib
211+
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<package_dir>/opencv/lib/:<package_dir>/deployment_tools/inference_engine/lib/<platform_type>/
212212
./object_detection_sample_ssd -m vehicle-license-plate-detection-barrier-0123.xml -i car_1.bmp -d CPU
213213
```
214214

‎modules/arm_plugin/armplg_build.sh‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,14 +29,27 @@ fail()
2929
exit $retval
3030
}
3131

32+
cloneSrcTree()
33+
{
34+
DESTDIR=$1
35+
shift
36+
SRCURL=$1
37+
shift
38+
while [ $# -gt 0 ]; do
39+
git lfs clone --recurse-submodules --shallow-submodules --depth 1 --branch=$1 $SRCURL $DESTDIR && return 0
40+
shift
41+
done
42+
return 1
43+
}
44+
3245
checkSrcTree()
3346
{
3447
[ $# -lt 3 ] && fail
3548

3649
if ! [ -d $1 ]; then
3750
echo "Unable to detect $1"
3851
echo "Cloning $2..."
39-
git lfs clone --recurse-submodules --shallow-submodules --depth 1 --branch=$3 $2 $1 || fail 3 "Failed to clone $2. Stopping"
52+
cloneSrcTree $@ || fail 3 "Failed to clone $2. Stopping"
4053
else
4154
echo "Detected $1"
4255
echo "Considering it as source directory"
@@ -45,7 +58,7 @@ checkSrcTree()
4558
echo "Removing existing sources..."
4659
rm -rf $1 || fail 1 "Failed to remove. Stopping"
4760
echo "Cloning $2..."
48-
git lfs clone --recurse-submodules --shallow-submodules --depth 1 --branch=$3 $2 $1 || fail 3 "Failed to clone $2. Stopping"
61+
cloneSrcTree $@ || fail 3 "Failed to clone $2. Stopping"
4962
elif [ -d $1/build ]; then
5063
echo "Build directory detected at $1"
5164
if [ "$UPDATE_SOURCES" = "clean" ]; then

‎modules/arm_plugin/src/transformations/arm_optimizations.cpp‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,7 @@ bool ArmPlugin::pass::ArmOptimizations::run_on_function(std::shared_ptr<ngraph::
110110
manager.register_pass<pass::ConvertConv1D>();
111111
manager.register_pass<pass::ConvertGroupConv1D>();
112112
manager.register_pass<pass::ConvertGroupConvolution>();
113+
manager.register_pass<ngraph::pass::ConstantFolding>();
113114
manager.register_pass<pass::ConvBiasActivationFusion>();
114115
manager.register_pass<pass::ConvertMatMulToFC>();
115116
manager.register_pass<pass::ConvertEltwise>();

‎modules/arm_plugin/src/transformations/conv_bias_activ_fusion.cpp‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@ ngraph::matcher_pass_callback ArmPlugin::pass::ConvBiasFusionBase::fuse_conv_wit
134134
}
135135

136136
if (!std::dynamic_pointer_cast<opset::Constant>(eltwise->input_value(1 - conv_idx).get_node_shared_ptr())) {
137-
IE_THROW() << "Unsupported Convolution with inconstant weights.";
137+
return false; // Unsupported Convolution with inconstant bias
138138
}
139139

140140
auto bias = eltwise->input_value(1 - conv_idx);
@@ -160,10 +160,6 @@ ngraph::matcher_pass_callback ArmPlugin::pass::ConvBiasFusionBase::fuse_conv_wit
160160
opset::Constant::create(ngraph::element::i64, ngraph::Shape{1}, {channel_dim}), true);
161161
}
162162

163-
if (m_conv->output(0).get_target_inputs().size() != 1) {
164-
return false;
165-
}
166-
167163
if (m_conv->inputs().size() == 3) {
168164
new_bias = std::make_shared<opset::Add>(new_bias, m_conv->input_value(Inputs::Bias));
169165
}
@@ -194,6 +190,10 @@ ngraph::matcher_pass_callback ArmPlugin::pass::ConvertConvBase::convert_conv_to_
194190
return false;
195191
}
196192

193+
if (!std::dynamic_pointer_cast<opset::Constant>(m_conv->input_value(Inputs::Weights).get_node_shared_ptr())) {
194+
IE_THROW() << "Unsupported Convolution with inconstant weights.";
195+
}
196+
197197
auto conv_arm = std::make_shared<ArmConv>(
198198
m_conv->input_value(Inputs::Data),
199199
m_conv->input_value(Inputs::Weights),
@@ -233,17 +233,17 @@ ArmPlugin::pass::ConvertGroupConvolutionToArm::ConvertGroupConvolutionToArm() {
233233

234234
ArmPlugin::pass::ConvBiasFusion::ConvBiasFusion() {
235235
auto m = std::make_shared<ngraph::pattern::Matcher>(
236-
ngraph::pattern::wrap_type<opset::ArmConvolution>({ngraph::pattern::any_input(ngraph::pattern::has_static_shape()),
237-
ngraph::pattern::any_input(ngraph::pattern::has_static_shape())},
238-
ngraph::pattern::has_static_shape()), "ConvBiasFusion");
236+
ngraph::pattern::wrap_type<opset::Add>({ngraph::pattern::any_input(ngraph::pattern::has_static_shape()),
237+
ngraph::pattern::any_input(ngraph::pattern::has_static_shape())},
238+
ngraph::pattern::has_static_shape()), "ConvBiasFusion");
239239
register_matcher(m, fuse_conv_with_bias<opset::ArmConvolution>());
240240
}
241241

242242
ArmPlugin::pass::GroupConvBiasFusion::GroupConvBiasFusion() {
243243
auto m = std::make_shared<ngraph::pattern::Matcher>(
244-
ngraph::pattern::wrap_type<opset::ArmGroupConvolution>({ngraph::pattern::any_input(ngraph::pattern::has_static_shape()),
245-
ngraph::pattern::any_input(ngraph::pattern::has_static_shape())},
246-
ngraph::pattern::has_static_shape()), "GroupConvBiasFusion");
244+
ngraph::pattern::wrap_type<opset::Add>({ngraph::pattern::any_input(ngraph::pattern::has_static_shape()),
245+
ngraph::pattern::any_input(ngraph::pattern::has_static_shape())},
246+
ngraph::pattern::has_static_shape()), "GroupConvBiasFusion");
247247
register_matcher(m, fuse_conv_with_bias<opset::ArmGroupConvolution>());
248248
}
249249

‎modules/java_api/cpp/blob.cpp‎

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -162,14 +162,15 @@ JNIEXPORT jlong JNICALL Java_org_intel_openvino_Blob_BlobLong(JNIEnv *env, jobje
162162
JNIEXPORT jlong JNICALL Java_org_intel_openvino_Blob_BlobCArray(JNIEnv *env, jobject obj, jlong tensorDescAddr, jlong matDataAddr)
163163
{
164164
static const char method_name[] = "BlobCArray";
165+
Blob::Ptr *blob = nullptr;
165166
try
166167
{
167168
TensorDesc *tDesc = (TensorDesc *)tensorDescAddr;
168169

169170
auto precision = tDesc->getPrecision();
170171

171172
std::vector<size_t> dims = tDesc->getDims();
172-
Blob::Ptr *blob = new Blob::Ptr();
173+
blob = new Blob::Ptr();
173174

174175
switch (precision) {
175176
case Precision::FP32:
@@ -210,18 +211,26 @@ JNIEXPORT jlong JNICALL Java_org_intel_openvino_Blob_BlobCArray(JNIEnv *env, job
210211
*blob = make_shared_blob<short>((*tDesc), data);
211212
break;
212213
}
213-
default:
214+
default: {
215+
delete blob;
214216
throw std::runtime_error("Unsupported precision value!");
217+
}
215218
}
216219

217220
return (jlong)blob;
218221
}
219222
catch (const std::exception &e)
220223
{
224+
if (blob) {
225+
delete blob;
226+
}
221227
throwJavaException(env, &e, method_name);
222228
}
223229
catch (...)
224230
{
231+
if (blob) {
232+
delete blob;
233+
}
225234
throwJavaException(env, 0, method_name);
226235
}
227236

@@ -258,8 +267,13 @@ JNIEXPORT jlong JNICALL Java_org_intel_openvino_Blob_rmap(JNIEnv *env, jobject o
258267
Blob::Ptr *output = reinterpret_cast<Blob::Ptr *>(addr);
259268

260269
if ((*output)->is<MemoryBlob>()) {
261-
LockedMemory<const void> *lmem = new LockedMemory<const void> (as<MemoryBlob>(*output)->rmap());
262-
return (jlong)lmem;
270+
auto mBlob = as<MemoryBlob>(*output);
271+
if (mBlob) {
272+
LockedMemory<const void> *lmem = new LockedMemory<const void> (mBlob->rmap());
273+
return (jlong)lmem;
274+
} else {
275+
throw std::runtime_error("Target Blob cannot be cast to the MemoryBlob!");
276+
}
263277
} else {
264278
throw std::runtime_error("Target Blob cannot be cast to the MemoryBlob!");
265279
}

‎modules/java_api/cpp/tensor_desc.cpp‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,9 @@ JNIEXPORT jintArray JNICALL Java_org_intel_openvino_TensorDesc_GetDims(JNIEnv *e
4545
std::vector<size_t> size_t_dims = tDesc->getDims();
4646

4747
jintArray result = env->NewIntArray(size_t_dims.size());
48+
if (!result) {
49+
throw std::runtime_error("Out of memory!");
50+
}
4851
jint *arr = env->GetIntArrayElements(result, nullptr);
4952

5053
for (int i = 0; i < size_t_dims.size(); ++i)

‎modules/java_api/org/intel/openvino/IEWrapper.java‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ protected long getNativeObjAddr() {
1414
@Override
1515
protected void finalize() throws Throwable {
1616
delete(nativeObj);
17+
super.finalize();
1718
}
1819

1920
/*----------------------------------- native methods -----------------------------------*/

‎modules/java_api/samples/ArgumentParser.java‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -25,24 +25,24 @@ private void printHelp() {
2525
}
2626

2727
public void parseArgs(String[] args) {
28-
try {
29-
for (int i = 0; i < args.length; i++) {
30-
String arg = args[i];
31-
if (arg.equals("--help") | arg.equals("-h")) {
32-
printHelp();
33-
System.exit(0);
34-
} else {
35-
if (description.containsKey(arg)) {
28+
for (int i = 0; i < args.length; i++) {
29+
String arg = args[i];
30+
if (arg.equals("--help") || arg.equals("-h")) {
31+
printHelp();
32+
System.exit(0);
33+
} else {
34+
if (description.containsKey(arg)) {
35+
if (i < args.length - 1) {
3636
input.put(arg, args[++i]);
3737
} else {
38-
System.out.println("Non-existent key: '" + arg + "'");
38+
System.out.println("Missed value for key: '" + arg + "'");
3939
System.exit(0);
4040
}
41+
} else {
42+
System.out.println("Non-existent key: '" + arg + "'");
43+
System.exit(0);
4144
}
4245
}
43-
} catch (ArrayIndexOutOfBoundsException e) {
44-
System.out.println("Error: Incorrect number of arguments");
45-
System.exit(0);
4646
}
4747
}
4848

‎modules/java_api/samples/benchmark_app/Main.java‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
import org.intel.openvino.*;
22

3+
import java.security.SecureRandom;
34
import java.util.ArrayList;
45
import java.util.Arrays;
56
import java.util.HashMap;
67
import java.util.Map;
7-
import java.util.Random;
88
import java.util.Vector;
99

1010
public class Main {
@@ -25,6 +25,9 @@ static boolean adjustShapesBatch(
2525
} else if (layout == Layout.CN) {
2626
batchIndex = 1;
2727
}
28+
if (!shapes.containsKey(entry.getKey())) {
29+
continue;
30+
}
2831
if ((batchIndex != -1) && (shapes.get(entry.getKey())[batchIndex] != batchSize)) {
2932
shapes.get(entry.getKey())[batchIndex] = batchSize;
3033
updated = true;
@@ -91,7 +94,7 @@ static Blob blobRandomByte(TensorDesc tDesc) {
9194
}
9295

9396
byte[] buff = new byte[size];
94-
Random rand = new Random();
97+
SecureRandom rand = new SecureRandom();
9598
rand.nextBytes(buff);
9699

97100
return new Blob(tDesc, buff);
@@ -108,7 +111,7 @@ static double getMedianValue(Vector<Double> vec) {
108111
else return (double) arr[arr.length / 2];
109112
}
110113

111-
static boolean getApiBoolean(String api) throws RuntimeException {
114+
static boolean getApiBoolean(String api) throws Exception {
112115
if (api.equals("sync")) return false;
113116
else if (api.equals("async")) return true;
114117
else throw new RuntimeException("Incorrect argument: '-api'");
@@ -193,7 +196,7 @@ public static void main(String[] args) {
193196

194197
try {
195198
isAsync = getApiBoolean(api);
196-
} catch (RuntimeException e) {
199+
} catch (Exception e) {
197200
System.out.println(e.getMessage());
198201
return;
199202
}
@@ -382,6 +385,10 @@ public static void main(String[] args) {
382385
|| (durationMs != 0L && execTime < durationMs)
383386
|| (isAsync && iteration % nireq != 0)) {
384387
inferRequest = inferRequestsQueue.getIdleRequest();
388+
if (inferRequest == null) {
389+
System.out.println("No idle Infer Requests!");
390+
return;
391+
}
385392

386393
if (isAsync) {
387394
// As the inference request is currently idle, the wait() adds no additional

0 commit comments

Comments
 (0)