From 01d3690c5ba4ca1308397ca198ce5142522efbe1 Mon Sep 17 00:00:00 2001
From: Zhanlue Yang
Date: Thu, 11 Nov 2021 13:30:01 +0800
Subject: [PATCH 01/35] Bug fix for layer_and_model documentation (#4069)
---
.../basic_concept/layer_and_model_cn.md | 20 +++++++++++++------
.../basic_concept/layer_and_model_en.md | 20 +++++++++++++------
2 files changed, 28 insertions(+), 12 deletions(-)
diff --git a/docs/guides/01_paddle2.0_introduction/basic_concept/layer_and_model_cn.md b/docs/guides/01_paddle2.0_introduction/basic_concept/layer_and_model_cn.md
index 9872d55e8f9..0f89e8308c4 100644
--- a/docs/guides/01_paddle2.0_introduction/basic_concept/layer_and_model_cn.md
+++ b/docs/guides/01_paddle2.0_introduction/basic_concept/layer_and_model_cn.md
@@ -303,8 +303,9 @@ Tensor(shape=[10, 1], dtype=float32, place=CPUPlace, stop_gradient=True,
同样的也可以使用 ``register_forward_pre_hook()`` 来注册**pre_hook**:
```python
-def forward_pre_hook(layer, input, output):
- return 2*output
+def forward_pre_hook(layer, input):
+ print(input)
+ return input
x = paddle.ones([10, 1], 'float32')
model = Model()
@@ -313,10 +314,17 @@ out = model(x)
```
```text
-Tensor(shape=[10, 1], dtype=float32, place=CPUPlace, stop_gradient=True,
- [[2.],
- [2.],
- ...
+(Tensor(shape=[10, 1], dtype=float32, place=CUDAPlace(0), stop_gradient=True,
+ [[1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.]]),)
```
## 模型数据保存
diff --git a/docs/guides/01_paddle2.0_introduction/basic_concept/layer_and_model_en.md b/docs/guides/01_paddle2.0_introduction/basic_concept/layer_and_model_en.md
index 3a667fc8c33..e96637cbb05 100644
--- a/docs/guides/01_paddle2.0_introduction/basic_concept/layer_and_model_en.md
+++ b/docs/guides/01_paddle2.0_introduction/basic_concept/layer_and_model_en.md
@@ -311,8 +311,9 @@ Tensor(shape=[10, 1], dtype=float32, place=CPUPlace, stop_gradient=True,
Similarly, we can also register a **pre_hook** through ``register_forward_pre_hook()``
```python
-def forward_pre_hook(layer, input, output):
- return 2*output
+def forward_pre_hook(layer, input):
+ print(input)
+ return input
x = paddle.ones([10, 1], 'float32')
model = Model()
@@ -321,10 +322,17 @@ out = model(x)
```
```text
-Tensor(shape=[10, 1], dtype=float32, place=CPUPlace, stop_gradient=True,
- [[2.],
- [2.],
- ...
+(Tensor(shape=[10, 1], dtype=float32, place=CUDAPlace(0), stop_gradient=True,
+ [[1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.],
+ [1.]]),)
```
## Save a model's data
From dfab71f05825eb5548c5f1237a41471c83063b49 Mon Sep 17 00:00:00 2001
From: Chen Long <1300851984@qq.com>
Date: Thu, 11 Nov 2021 14:39:32 +0800
Subject: [PATCH 02/35] update docs (#4067)
* update docs
* update docs
* mv multi_dot to linalg
* update styles
* fix ocr
---
.../paddle/{ => linalg}/matrix_power_cn.rst | 10 +-
docs/api/paddle/{ => linalg}/multi_dot_cn.rst | 2 +-
.../basic_concept/amp_cn.ipynb | 463 +++++++++++
.../basic_concept/amp_cn.md | 42 +-
.../basic_concept/amp_en.ipynb | 453 +++++++++++
.../basic_concept/amp_en.md | 88 ++-
.../basic_concept/autograd_cn.rst | 4 +-
.../basic_concept/gradient_clip_cn.rst | 2 +
.../basic_concept/gradient_clip_en.rst | 2 +
.../basic_concept/tensor_introduction_cn.md | 5 +-
.../basic_concept/tensor_introduction_en.md | 5 +-
...model.rst => load_old_format_model_cn.rst} | 0
.../migration_cn.rst | 2 +-
.../01_paddle2.0_introduction/update_cn.md | 4 +-
.../05_train_eval_predict_cn.rst | 112 +--
.../guides/performance_improving/index_cn.rst | 7 +-
docs/install/docker/fromdocker.rst | 1 -
docs/install/docker/fromdocker_en.rst | 1 -
docs/practices/cv/image_ocr.ipynb | 722 +++++++++++++++++
docs/practices/cv/image_ocr/image_ocr.ipynb | 739 ------------------
docs/practices/cv/image_ocr/images/image1.png | Bin 133262 -> 0 bytes
docs/practices/cv/image_ocr/images/image2.png | Bin 93944 -> 0 bytes
docs/practices/cv/image_ocr/images/image3.png | Bin 508150 -> 0 bytes
docs/practices/cv/index_cn.rst | 6 +-
.../cv/{image_ocr => }/sample_img/9450.jpg | Bin
.../cv/{image_ocr => }/sample_img/9451.jpg | Bin
.../cv/{image_ocr => }/sample_img/9452.jpg | Bin
docs/practices/index_cn.rst | 2 +-
docs/release_note_cn.md | 123 ++-
docs/release_note_en.md | 116 ++-
30 files changed, 1968 insertions(+), 943 deletions(-)
rename docs/api/paddle/{ => linalg}/matrix_power_cn.rst (86%)
rename docs/api/paddle/{ => linalg}/multi_dot_cn.rst (97%)
create mode 100644 docs/guides/01_paddle2.0_introduction/basic_concept/amp_cn.ipynb
create mode 100644 docs/guides/01_paddle2.0_introduction/basic_concept/amp_en.ipynb
rename docs/guides/01_paddle2.0_introduction/{load_old_format_model.rst => load_old_format_model_cn.rst} (100%)
create mode 100644 docs/practices/cv/image_ocr.ipynb
delete mode 100644 docs/practices/cv/image_ocr/image_ocr.ipynb
delete mode 100644 docs/practices/cv/image_ocr/images/image1.png
delete mode 100644 docs/practices/cv/image_ocr/images/image2.png
delete mode 100644 docs/practices/cv/image_ocr/images/image3.png
rename docs/practices/cv/{image_ocr => }/sample_img/9450.jpg (100%)
rename docs/practices/cv/{image_ocr => }/sample_img/9451.jpg (100%)
rename docs/practices/cv/{image_ocr => }/sample_img/9452.jpg (100%)
diff --git a/docs/api/paddle/matrix_power_cn.rst b/docs/api/paddle/linalg/matrix_power_cn.rst
similarity index 86%
rename from docs/api/paddle/matrix_power_cn.rst
rename to docs/api/paddle/linalg/matrix_power_cn.rst
index 210b41e61c9..c1f771a92f0 100644
--- a/docs/api/paddle/matrix_power_cn.rst
+++ b/docs/api/paddle/linalg/matrix_power_cn.rst
@@ -3,7 +3,7 @@
matrix_power
-------------------------------
-.. py:function:: paddle.matrix_power(x, n, name=None)
+.. py:function:: paddle.linalg.matrix_power(x, n, name=None)
计算一个或一批方阵的 ``n`` 次幂。
@@ -41,17 +41,17 @@ matrix_power
x = paddle.to_tensor([[1, 2, 3],
[1, 4, 9],
[1, 8, 27]], dtype='float64')
- print(paddle.matrix_power(x, 2))
+ print(paddle.linalg.matrix_power(x, 2))
# [[6. , 34. , 102.],
# [14. , 90. , 282.],
# [36. , 250., 804.]]
- print(paddle.matrix_power(x, 0))
+ print(paddle.linalg.matrix_power(x, 0))
# [[1., 0., 0.],
# [0., 1., 0.],
# [0., 0., 1.]]
- print(paddle.matrix_power(x, -2))
+ print(paddle.linalg.matrix_power(x, -2))
# [[ 12.91666667, -12.75000000, 2.83333333 ],
# [-7.66666667 , 8. , -1.83333333 ],
- # [ 1.80555556 , -1.91666667 , 0.44444444 ]]
\ No newline at end of file
+ # [ 1.80555556 , -1.91666667 , 0.44444444 ]]
diff --git a/docs/api/paddle/multi_dot_cn.rst b/docs/api/paddle/linalg/multi_dot_cn.rst
similarity index 97%
rename from docs/api/paddle/multi_dot_cn.rst
rename to docs/api/paddle/linalg/multi_dot_cn.rst
index 8dc63f4a419..e6200eecbdd 100755
--- a/docs/api/paddle/multi_dot_cn.rst
+++ b/docs/api/paddle/linalg/multi_dot_cn.rst
@@ -3,7 +3,7 @@
multi_dot
-------------------------------
-.. py:function:: paddle.multi_dot(x, name=None)
+.. py:function:: paddle.linalg.multi_dot(x, name=None)
Multi_dot是一个计算多个矩阵乘法的算子。
diff --git a/docs/guides/01_paddle2.0_introduction/basic_concept/amp_cn.ipynb b/docs/guides/01_paddle2.0_introduction/basic_concept/amp_cn.ipynb
new file mode 100644
index 00000000000..e5a5b2106b8
--- /dev/null
+++ b/docs/guides/01_paddle2.0_introduction/basic_concept/amp_cn.ipynb
@@ -0,0 +1,463 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "# 自动混合精度训练\n",
+ "\n",
+ "一般情况下,训练深度学习模型时使用的数据类型为单精度(FP32)。2018年,百度与NVIDIA联合发表论文:[MIXED PRECISION TRAINING](https://arxiv.org/pdf/1710.03740.pdf),提出了混合精度训练的方法。混合精度训练是指在训练过程中,同时使用单精度(FP32)和半精度(FP16),其目的是相较于使用单精度(FP32)训练模型,在保持精度持平的条件下,能够加速训练。本文将介绍如何使用飞桨框架,实现自动混合精度训练。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 一、半精度浮点类型 FP16\n",
+ "\n",
+ "首先介绍半精度(FP16)。如图1所示,半精度(FP16)是一种相对较新的浮点类型,在计算机中使用2字节(16位)存储。在IEEE 754-2008标准中,它亦被称作binary16。与计算中常用的单精度(FP32)和双精度(FP64)类型相比,FP16更适于在精度要求不高的场景中使用。\n",
+ "\n",
+ "\n",
+ "
\n",
+ " 图 1. 半精度和单精度数据示意图\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 二、NVIDIA GPU的FP16算力\n",
+ "在使用相同的超参数下,混合精度训练使用半精度浮点(FP16)和单精度(FP32)浮点即可达到与使用纯单精度训练相同的准确率,并可加速模型的训练速度。这主要得益于英伟达推出的Volta及Turing架构GPU在使用FP16计算时具有如下特点:\n",
+ "- FP16可降低一半的内存带宽和存储需求,这使得在相同的硬件条件下研究人员可使用更大更复杂的模型以及更大的batch size大小。\n",
+ "- FP16可以充分利用英伟达Volta及Turing架构GPU提供的Tensor Cores技术。在相同的GPU硬件上,Tensor Cores的FP16计算吞吐量是FP32的8倍。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 三、使用飞桨框架实现自动混合精度\n",
+ "使用飞桨框架提供的API,``paddle.amp.auto_cast`` 和 ``paddle.amp.decorate`` 和 ``paddle.amp.GradScaler`` 能够实现自动混合精度训练(Automatic Mixed Precision,AMP),即在相关OP的计算中,根据一定的规则,自动选择FP16或FP32计算。飞桨的AMP为用户提供了两种模式:\n",
+ "- level=’O1‘:采用黑名名单策略的混合精度训练,使用FP16与FP32进行计算的OP列表可见该[文档](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/amp/Overview_cn.html)。\n",
+ "- level=’O2‘:纯FP16训练,除用户自定义黑名单中指定的OP和不支持FP16计算的OP之外,全部使用FP16计算。\n",
+ "\n",
+ "下面来看一个具体的例子,来了解如果使用飞桨框架实现混合精度训练。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 3.1 辅助函数\n",
+ "首先定义辅助函数,用来计算训练时间。"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import time\n",
+ "\n",
+ "# 开始时间\n",
+ "start_time = None\n",
+ "\n",
+ "def start_timer():\n",
+ " # 获取开始时间\n",
+ " global start_time\n",
+ " start_time = time.time()\n",
+ "\n",
+ "def end_timer_and_print(msg):\n",
+ " # 打印信息并输出训练时间\n",
+ " end_time = time.time()\n",
+ " print(\"\\n\" + msg)\n",
+ " print(\"共计耗时 = {:.3f} sec\".format(end_time - start_time))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 3.2 构建一个简单的网络\n",
+ "\n",
+ "构建一个简单的网络,用于对比使用普通方法进行训练与使用混合精度训练的训练速度。该网络由三层 ``Linear`` 组成,其中前两层 ``Linear`` 后接 ``ReLU`` 激活函数。"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "import paddle.nn as nn\n",
+ "\n",
+ "class SimpleNet(nn.Layer):\n",
+ "\n",
+ " def __init__(self, input_size, output_size):\n",
+ " \n",
+ " super(SimpleNet, self).__init__()\n",
+ " self.linear1 = nn.Linear(input_size, output_size)\n",
+ " self.relu1 = nn.ReLU()\n",
+ " self.linear2 = nn.Linear(input_size, output_size)\n",
+ " self.relu2 = nn.ReLU()\n",
+ " self.linear3 = nn.Linear(input_size, output_size)\n",
+ "\n",
+ " def forward(self, x):\n",
+ "\n",
+ " x = self.linear1(x)\n",
+ " x = self.relu1(x)\n",
+ " x = self.linear2(x)\n",
+ " x = self.relu2(x)\n",
+ " x = self.linear3(x)\n",
+ "\n",
+ " return x"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "设置训练的相关参数,这里为了能有效的看出混合精度训练对于训练速度的提升,将 ``input_size`` 与 ``output_size`` 的值设为较大的值,为了使用GPU 提供的``Tensor Core`` 性能,还需将 ``batch_size`` 设置为 8 的倍数。"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "W1110 18:42:02.362493 104 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1\n",
+ "W1110 18:42:02.367755 104 device_context.cc:465] device: 0, cuDNN Version: 7.6.\n"
+ ]
+ }
+ ],
+ "source": [
+ "epochs = 5\n",
+ "input_size = 4096 # 设为较大的值\n",
+ "output_size = 4096 # 设为较大的值\n",
+ "batch_size = 512 # batch_size 为8的倍数\n",
+ "nums_batch = 50\n",
+ "\n",
+ "train_data = [paddle.randn((batch_size, input_size)) for _ in range(nums_batch)]\n",
+ "labels = [paddle.randn((batch_size, output_size)) for _ in range(nums_batch)]\n",
+ "\n",
+ "mse = paddle.nn.MSELoss()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 3.3 使用默认的训练方式进行训练"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
+ " [1.24519622])\n",
+ "\n",
+ "默认耗时:\n",
+ "共计耗时 = 2.926 sec\n"
+ ]
+ }
+ ],
+ "source": [
+ "model = SimpleNet(input_size, output_size) # 定义模型\n",
+ "\n",
+ "optimizer = paddle.optimizer.SGD(learning_rate=0.0001, parameters=model.parameters()) # 定义优化器\n",
+ "\n",
+ "start_timer() # 获取训练开始时间\n",
+ "\n",
+ "for epoch in range(epochs):\n",
+ " datas = zip(train_data, labels)\n",
+ " for i, (data, label) in enumerate(datas):\n",
+ "\n",
+ " output = model(data)\n",
+ " loss = mse(output, label)\n",
+ "\n",
+ " # 反向传播\n",
+ " loss.backward()\n",
+ "\n",
+ " # 训练模型\n",
+ " optimizer.step()\n",
+ " optimizer.clear_grad()\n",
+ "\n",
+ "print(loss)\n",
+ "end_timer_and_print(\"默认耗时:\") # 获取结束时间并打印相关信息"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 3.4 使用AMP训练模型\n",
+ "\n",
+ "在飞桨框架中,使用自动混合精度训练,需要进行四个步骤:\n",
+ "\n",
+ "- Step1: 定义 ``GradScaler`` ,用于缩放 ``loss`` 比例,避免浮点数下溢\n",
+ "- Step2: 使用 ``decorate`` 在level=’O1‘模式下不做任何处理,无需调用该api,在level=’O2‘模式下,将网络参数从FP32转换为FP16\n",
+ "- Step3: 使用 ``auto_cast`` 用于创建AMP上下文环境,该上下文中自动会确定每个OP的输入数据类型(FP16或FP32)\n",
+ "- Step4: 使用 Step1中定义的 ``GradScaler`` 完成 ``loss`` 的缩放,用缩放后的 ``loss`` 进行反向传播,完成训练\n",
+ "\n",
+ "\n",
+ "采用level=’O1‘模式训练:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
+ " [1.24815702])\n",
+ "\n",
+ "使用AMP-O1模式耗时:\n",
+ "共计耗时 = 1.294 sec\n"
+ ]
+ }
+ ],
+ "source": [
+ "model = SimpleNet(input_size, output_size) # 定义模型\n",
+ "\n",
+ "optimizer = paddle.optimizer.SGD(learning_rate=0.0001, parameters=model.parameters()) # 定义优化器\n",
+ "\n",
+ "# Step1:定义 GradScaler,用于缩放loss比例,避免浮点数溢出\n",
+ "scaler = paddle.amp.GradScaler(init_loss_scaling=1024)\n",
+ "\n",
+ "start_timer() # 获取训练开始时间\n",
+ "\n",
+ "for epoch in range(epochs):\n",
+ " datas = zip(train_data, labels)\n",
+ " for i, (data, label) in enumerate(datas):\n",
+ "\n",
+ " # Step2:创建AMP上下文环境,开启自动混合精度训练\n",
+ " with paddle.amp.auto_cast():\n",
+ " output = model(data)\n",
+ " loss = mse(output, label)\n",
+ "\n",
+ " # Step3:使用 Step1中定义的 GradScaler 完成 loss 的缩放,用缩放后的 loss 进行反向传播\n",
+ " scaled = scaler.scale(loss)\n",
+ " scaled.backward()\n",
+ "\n",
+ " # 训练模型\n",
+ " scaler.minimize(optimizer, scaled)\n",
+ " optimizer.clear_grad()\n",
+ "\n",
+ "print(loss)\n",
+ "end_timer_and_print(\"使用AMP-O1模式耗时:\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "采用level=’O2‘模式训练:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "in ParamBase copy_to func\n",
+ "in ParamBase copy_to func\n",
+ "in ParamBase copy_to func\n",
+ "in ParamBase copy_to func\n",
+ "in ParamBase copy_to func\n",
+ "in ParamBase copy_to func\n",
+ "Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
+ " [1.25423336])\n",
+ "\n",
+ "使用AMP-O2模式耗时:\n",
+ "共计耗时 = 0.890 sec\n"
+ ]
+ }
+ ],
+ "source": [
+ "model = SimpleNet(input_size, output_size) # 定义模型\n",
+ "\n",
+ "optimizer = paddle.optimizer.SGD(learning_rate=0.0001, parameters=model.parameters()) # 定义优化器\n",
+ "\n",
+ "# Step1:定义 GradScaler,用于缩放loss比例,避免浮点数溢出\n",
+ "scaler = paddle.amp.GradScaler(init_loss_scaling=1024)\n",
+ "\n",
+ "# Step2:在level=’O2‘模式下,将网络参数从FP32转换为FP16\n",
+ "model, optimizer = paddle.amp.decorate(models=model, optimizers=optimizer, level='O2', master_weight=None, save_dtype=None)\n",
+ "\n",
+ "start_timer() # 获取训练开始时间\n",
+ "\n",
+ "for epoch in range(epochs):\n",
+ " datas = zip(train_data, labels)\n",
+ " for i, (data, label) in enumerate(datas):\n",
+ "\n",
+ " # Step3:创建AMP上下文环境,开启自动混合精度训练\n",
+ " with paddle.amp.auto_cast(enable=True, custom_white_list=None, custom_black_list=None, level='O2'):\n",
+ " output = model(data)\n",
+ " loss = mse(output, label)\n",
+ "\n",
+ " # Step4:使用 Step1中定义的 GradScaler 完成 loss 的缩放,用缩放后的 loss 进行反向传播\n",
+ " scaled = scaler.scale(loss)\n",
+ " scaled.backward()\n",
+ "\n",
+ " # 训练模型\n",
+ " scaler.minimize(optimizer, scaled)\n",
+ " optimizer.clear_grad()\n",
+ "\n",
+ "print(loss)\n",
+ "end_timer_and_print(\"使用AMP-O2模式耗时:\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 四、进阶用法\n",
+ "### 4.1 使用梯度累加\n",
+ "梯度累加是指在模型训练过程中,训练一个batch的数据得到梯度后,不立即用该梯度更新模型参数,而是继续下一个batch数据的训练,得到梯度后继续循环,多次循环后梯度不断累加,直至达到一定次数后,用累加的梯度更新参数,这样可以起到变相扩大 batch_size 的作用。\n",
+ "\n",
+ "在自动混合精度训练中,也支持梯度累加,使用方式如下:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
+ " [1.25602019])\n",
+ "\n",
+ "使用AMP模式耗时:\n",
+ "共计耗时 = 1.026 sec\n"
+ ]
+ }
+ ],
+ "source": [
+ "model = SimpleNet(input_size, output_size) # 定义模型\n",
+ "\n",
+ "optimizer = paddle.optimizer.SGD(learning_rate=0.0001, parameters=model.parameters()) # 定义优化器\n",
+ "\n",
+ "accumulate_batchs_num = 10 # 梯度累加中 batch 的数量\n",
+ "\n",
+ "# 定义 GradScaler\n",
+ "scaler = paddle.amp.GradScaler(init_loss_scaling=1024)\n",
+ "\n",
+ "start_timer() # 获取训练开始时间\n",
+ "\n",
+ "for epoch in range(epochs):\n",
+ " datas = zip(train_data, labels)\n",
+ " for i, (data, label) in enumerate(datas):\n",
+ "\n",
+ " # 创建AMP上下文环境,开启自动混合精度训练\n",
+ " with paddle.amp.auto_cast():\n",
+ " output = model(data)\n",
+ " loss = mse(output, label)\n",
+ "\n",
+ " # 使用 GradScaler 完成 loss 的缩放,用缩放后的 loss 进行反向传播\n",
+ " scaled = scaler.scale(loss)\n",
+ " scaled.backward()\n",
+ "\n",
+ " # 当累计的 batch 为 accumulate_batchs_num 时,更新模型参数\n",
+ " if (i + 1) % accumulate_batchs_num == 0:\n",
+ "\n",
+ " # 训练模型\n",
+ " scaler.minimize(optimizer, scaled)\n",
+ " optimizer.clear_grad()\n",
+ "\n",
+ "print(loss)\n",
+ "end_timer_and_print(\"使用AMP模式耗时:\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 五、总结\n",
+ "从上面的示例中可以看出,使用自动混合精度训练,O1模式共计耗时约 1.294s,O2模式共计耗时约 0.890s,而普通的训练方式则耗时 2.926s,O1模式训练速度提升约为 2.1倍,O2模式训练速度提升约为 3.0倍。如需更多使用混合精度训练的示例,请参考飞桨模型库: [paddlepaddle/models](https://github.com/PaddlePaddle/models)。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "py35-paddle1.2.0"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.4"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}
diff --git a/docs/guides/01_paddle2.0_introduction/basic_concept/amp_cn.md b/docs/guides/01_paddle2.0_introduction/basic_concept/amp_cn.md
index bc96b6736a4..646e01ecd37 100644
--- a/docs/guides/01_paddle2.0_introduction/basic_concept/amp_cn.md
+++ b/docs/guides/01_paddle2.0_introduction/basic_concept/amp_cn.md
@@ -1,6 +1,6 @@
# 自动混合精度训练
-一般情况下,训练深度学习模型时使用的数据类型为单精度(FP32)。2018年,百度与NVIDIA联合发表论文:[MIXED PRECISION TRAINING](https://arxiv.org/pdf/1710.03740.pdf),提出了混合精度训练的方法。混合精度训练是指在训练过程中,同时使用单精度(FP32)和半精度(FP16),其目的是相较于使用单精度(FP32)训练模型,在保持精度持平的条件下,能够加速训练。本文将介绍如何使用飞桨框架,实现自动混合精度训练。
+一般情况下,训练深度学习模型时使用的数据类型为单精度(FP32)。2018年,百度与NVIDIA联合发表论文:[MIXED PRECISION TRAINING](https://arxiv.org/pdf/1710.03740.pdf),提出了混合精度训练的方法。混合精度训练是指在训练过程中,同时使用单精度(FP32)和半精度(FP16),其目的是相较于使用单精度(FP32)训练模型,在保持精度持平的条件下,能够加速训练。本文将介绍如何使用飞桨框架,实现自动混合精度训练。
## 一、半精度浮点类型 FP16
@@ -57,6 +57,7 @@ import paddle.nn as nn
class SimpleNet(nn.Layer):
def __init__(self, input_size, output_size):
+
super(SimpleNet, self).__init__()
self.linear1 = nn.Linear(input_size, output_size)
self.relu1 = nn.ReLU()
@@ -91,6 +92,10 @@ labels = [paddle.randn((batch_size, output_size)) for _ in range(nums_batch)]
mse = paddle.nn.MSELoss()
```
+ W1110 18:42:02.362493 104 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
+ W1110 18:42:02.367755 104 device_context.cc:465] device: 0, cuDNN Version: 7.6.
+
+
### 3.3 使用默认的训练方式进行训练
@@ -120,10 +125,10 @@ end_timer_and_print("默认耗时:") # 获取结束时间并打印相关信息
```
Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
- [1.24609220])
-
+ [1.24519622])
+
默认耗时:
- 共计耗时 = 2.819 sec
+ 共计耗时 = 2.926 sec
### 3.4 使用AMP训练模型
@@ -138,6 +143,7 @@ end_timer_and_print("默认耗时:") # 获取结束时间并打印相关信息
采用level=’O1‘模式训练:
+
```python
model = SimpleNet(input_size, output_size) # 定义模型
@@ -170,14 +176,15 @@ end_timer_and_print("使用AMP-O1模式耗时:")
```
Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
- [1.24609900])
-
+ [1.24815702])
+
使用AMP-O1模式耗时:
- 共计耗时 = 1.324 sec
+ 共计耗时 = 1.294 sec
采用level=’O2‘模式训练:
+
```python
model = SimpleNet(input_size, output_size) # 定义模型
@@ -212,11 +219,17 @@ print(loss)
end_timer_and_print("使用AMP-O2模式耗时:")
```
+ in ParamBase copy_to func
+ in ParamBase copy_to func
+ in ParamBase copy_to func
+ in ParamBase copy_to func
+ in ParamBase copy_to func
+ in ParamBase copy_to func
Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
- [1.24997652])
-
+ [1.25423336])
+
使用AMP-O2模式耗时:
- 共计耗时 = 0.933 sec
+ 共计耗时 = 0.890 sec
## 四、进阶用法
@@ -263,10 +276,11 @@ end_timer_and_print("使用AMP模式耗时:")
```
Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
- [1.24623466])
-
+ [1.25602019])
+
使用AMP模式耗时:
- 共计耗时 = 1.020 sec
+ 共计耗时 = 1.026 sec
+
## 五、总结
-从上面的示例中可以看出,使用自动混合精度训练,O1模式共计耗时约 1.324s,O2模式共计耗时约 0.933s,而普通的训练方式则耗时 2.819s,O1模式训练速度提升约为 2.1倍,O2模式训练速度提升约为 3.0倍。如需更多使用混合精度训练的示例,请参考飞桨模型库: [paddlepaddle/models](https://github.com/PaddlePaddle/models)。
+从上面的示例中可以看出,使用自动混合精度训练,O1模式共计耗时约 1.294s,O2模式共计耗时约 0.890s,而普通的训练方式则耗时 2.926s,O1模式训练速度提升约为 2.1倍,O2模式训练速度提升约为 3.0倍。如需更多使用混合精度训练的示例,请参考飞桨模型库: [paddlepaddle/models](https://github.com/PaddlePaddle/models)。
diff --git a/docs/guides/01_paddle2.0_introduction/basic_concept/amp_en.ipynb b/docs/guides/01_paddle2.0_introduction/basic_concept/amp_en.ipynb
new file mode 100644
index 00000000000..22c12fcfed1
--- /dev/null
+++ b/docs/guides/01_paddle2.0_introduction/basic_concept/amp_en.ipynb
@@ -0,0 +1,453 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "# Automatic Mixed Precision Training\n",
+ "\n",
+ "In general, the datatype of training deep learning models is single-precision floating-point format(also called FP32). In 2018, Baidu and NVIDIA jointly published the paper: [MIXED PRECISION TRAINING](https://arxiv.org/pdf/1710.03740.pdf), which proposed mixed precision training. During the process of training, some operators use FP32 and other operators use half precision(also called FP16) in the same time. Its purpose is to speed up training, while compared with the FP32 training model, the same accuracy is maintained. This tutorial will introduce how to use automatic mixed precision training with PaddlePaddle."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 1. Half Precision (FP16)\n",
+ "\n",
+ "First introduce FP16. As shown in Figure 1, FP16 occupies 16 bits (two bytes in modern computers) of computer memory. In the IEEE 754-2008 standard, it is also named binary16. Compared with FP32 and double precision (also called FP64) commonly used, FP16 is more suitable for the usage in scenarios with low precision requirements.\n",
+ "\n",
+ "\n",
+ "
\n",
+ " Figure 1. Half precision(FP16) and single precision(FP32)\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 2. FP16 Computing Power of NVIDIA GPU\n",
+ "\n",
+ "When the same hyperparameters are used, mixed precision training using FP16 and FP32 can achieve the same accuracy as that of pure single precision used, and can accelerate the training speed. It mainly attributes to the features that NVIDIA Volta and NVIDIA Turing use FP16 to calculate:\n",
+ "- FP16 can reduce memory bandwidth and storage requirements by half, which allows researchers to use more complex models and larger batch sizes under the same hardware conditions.\n",
+ "- FP16 can make full use of Tensor Cores technology provided by NVIDIA Volta and NVIDIA Turing. On the same GPU hardware, the computing throughput of Tensor Cores' FP16 is 8 times bigger than that of FP32."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 3. Automatic Mixed Precision Training with PaddlePaddle\n",
+ "\n",
+ "Using PaddlePaddle's API ``paddle.amp.auto_cast`` and ``paddle.amp.GradScaler`` can realize automatic mixed precision training (AMP), which can automatically choose FP16 or FP32 for different operators' calculation. After the AMP mode is turned on, the operator list calculated by FP16 and FP32 can be found in this [document](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/amp/Overview_cn.html). This is a specific example to understand how to use PaddlePaddle to achieve mixed precision training."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 3.1 Auxiliary Function\n",
+ "First define the auxiliary function to calculate the training time."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import time\n",
+ "\n",
+ "# start time\n",
+ "start_time = None\n",
+ "\n",
+ "def start_timer():\n",
+ " # get start time\n",
+ " global start_time\n",
+ " start_time = time.time()\n",
+ "\n",
+ "def end_timer_and_print(msg):\n",
+ " # print message and total training time\n",
+ " end_time = time.time()\n",
+ " print(\"\\n\" + msg)\n",
+ " print(\"total time = {:.3f} sec\".format(end_time - start_time))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 3.2 A Simple Network\n",
+ "\n",
+ "Define a simple network to compare the training speed of common methods and mixed precision. The network is composed of three layers of ``Linear``. The first two layers of ``Linear`` are followed by the ``ReLU`` activation function."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "import paddle.nn as nn\n",
+ "\n",
+ "class SimpleNet(nn.Layer):\n",
+ "\n",
+ " def __init__(self, input_size, output_size):\n",
+ " \n",
+ " super(SimpleNet, self).__init__()\n",
+ " self.linear1 = nn.Linear(input_size, output_size)\n",
+ " self.relu1 = nn.ReLU()\n",
+ " self.linear2 = nn.Linear(input_size, output_size)\n",
+ " self.relu2 = nn.ReLU()\n",
+ " self.linear3 = nn.Linear(input_size, output_size)\n",
+ "\n",
+ " def forward(self, x):\n",
+ "\n",
+ " x = self.linear1(x)\n",
+ " x = self.relu1(x)\n",
+ " x = self.linear2(x)\n",
+ " x = self.relu2(x)\n",
+ " x = self.linear3(x)\n",
+ "\n",
+ " return x"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "Set the parameters of training. In order to effectively show the improvement of training speed by mixed precision training, please set the larger values of ``input_size`` and ``output_size``. And in order to use the ``Tensor Core`` provided by GPU, ``batch_size`` needs to be set as a multiple of 8."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "epochs = 5\n",
+ "input_size = 4096 # set to a larger value\n",
+ "output_size = 4096 # set to a larger value\n",
+ "batch_size = 512 # batch_size is a multiple of 8\n",
+ "nums_batch = 50\n",
+ "\n",
+ "train_data = [paddle.randn((batch_size, input_size)) for _ in range(nums_batch)]\n",
+ "labels = [paddle.randn((batch_size, output_size)) for _ in range(nums_batch)]\n",
+ "\n",
+ "mse = paddle.nn.MSELoss()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 3.3 Training with Default Method"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
+ " [1.24072289])\n",
+ "\n",
+ "Default time:\n",
+ "total time = 2.935 sec\n"
+ ]
+ }
+ ],
+ "source": [
+ "model = SimpleNet(input_size, output_size) # define model\n",
+ "\n",
+ "optimizer = paddle.optimizer.SGD(learning_rate=0.0001, parameters=model.parameters()) # define optimizer\n",
+ "\n",
+ "start_timer() # get the start time of training\n",
+ "\n",
+ "for epoch in range(epochs):\n",
+ " datas = zip(train_data, labels)\n",
+ " for i, (data, label) in enumerate(datas):\n",
+ "\n",
+ " output = model(data)\n",
+ " loss = mse(output, label)\n",
+ "\n",
+ " # backpropagation\n",
+ " loss.backward()\n",
+ "\n",
+ " # update parameters\n",
+ " optimizer.step()\n",
+ " optimizer.clear_grad()\n",
+ "\n",
+ "print(loss)\n",
+ "end_timer_and_print(\"Default time:\") # print massage and total time"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 3.4 Training with AMP\n",
+ "\n",
+ "Using automatic mixed precision training with PaddlePaddle requires four steps:\n",
+ "\n",
+ "- Step1: Define ``GradScaler``, which is used to scale the ``loss`` to avoid underflow\n",
+ "- Step2: Use ``decorate``, to do nothing in level='O1' mode without using this api, and in level='O2' mode to convert network parameters from FP32 to FP16\n",
+ "- Step3: Use ``auto_cast`` to create an AMP context, in which the input datatype(FP16 or FP32) of each oprator will be automatically determined\n",
+ "- Step4: Use ``GradScaler`` defined in Step1 to complete the scaling of ``loss``, and use the scaled ``loss`` for backpropagation to complete the training\n",
+ "\n",
+ "In level=’O1‘ mode:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
+ " [1.24848151])\n",
+ "\n",
+ "AMP time in O1 mode:\n",
+ "total time = 1.299 sec\n"
+ ]
+ }
+ ],
+ "source": [
+ "model = SimpleNet(input_size, output_size) # define model\n",
+ "\n",
+ "optimizer = paddle.optimizer.SGD(learning_rate=0.0001, parameters=model.parameters()) # define optimizer\n",
+ "\n",
+ "# Step1:define GradScaler\n",
+ "scaler = paddle.amp.GradScaler(init_loss_scaling=1024)\n",
+ "\n",
+ "start_timer() # get start time\n",
+ "\n",
+ "for epoch in range(epochs):\n",
+ " datas = zip(train_data, labels)\n",
+ " for i, (data, label) in enumerate(datas):\n",
+ "\n",
+ " # Step2:create AMP context environment\n",
+ " with paddle.amp.auto_cast():\n",
+ " output = model(data)\n",
+ " loss = mse(output, label)\n",
+ "\n",
+ " # Step3:use GradScaler complete the loss scaling\n",
+ " scaled = scaler.scale(loss)\n",
+ " scaled.backward()\n",
+ "\n",
+ " # update parameters\n",
+ " scaler.minimize(optimizer, scaled)\n",
+ " optimizer.clear_grad()\n",
+ "\n",
+ "print(loss)\n",
+ "end_timer_and_print(\"AMP time in O1 mode:\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "In level='O2' mode:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 30,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "in ParamBase copy_to func\n",
+ "in ParamBase copy_to func\n",
+ "in ParamBase copy_to func\n",
+ "in ParamBase copy_to func\n",
+ "in ParamBase copy_to func\n",
+ "in ParamBase copy_to func\n",
+ "Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
+ " [1.25075114])\n",
+ "\n",
+ "AMP time in O2 mode:\n",
+ "total time = 0.888 sec\n"
+ ]
+ }
+ ],
+ "source": [
+ "model = SimpleNet(input_size, output_size) # define model\n",
+ "\n",
+ "optimizer = paddle.optimizer.SGD(learning_rate=0.0001, parameters=model.parameters()) # define optimizer\n",
+ "\n",
+ "# Step1:define GradScaler\n",
+ "scaler = paddle.amp.GradScaler(init_loss_scaling=1024)\n",
+ "\n",
+ "# Step2:in level='O2' mode, convert network parameters from FP32 to FP16\n",
+ "model, optimizer = paddle.amp.decorate(models=model, optimizers=optimizer, level='O2', master_weight=None, save_dtype=None)\n",
+ "\n",
+ "start_timer() # get start time\n",
+ "\n",
+ "for epoch in range(epochs):\n",
+ " datas = zip(train_data, labels)\n",
+ " for i, (data, label) in enumerate(datas):\n",
+ "\n",
+ " # Step3:create AMP context environment\n",
+ " with paddle.amp.auto_cast(enable=True, custom_white_list=None, custom_black_list=None, level='O2'):\n",
+ " output = model(data)\n",
+ " loss = mse(output, label)\n",
+ "\n",
+ " # Step4:use GradScaler complete the loss scaling\n",
+ " scaled = scaler.scale(loss)\n",
+ " scaled.backward()\n",
+ "\n",
+ " # update parameters\n",
+ " scaler.minimize(optimizer, scaled)\n",
+ " optimizer.clear_grad()\n",
+ "\n",
+ "print(loss)\n",
+ "end_timer_and_print(\"AMP time in O2 mode:\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 4. Advanced Usage\n",
+ "### 4.1 Gradient Accumulation\n",
+ "\n",
+ "Gradient accumulation means running a configured number of steps without updating the model variables. Until certain steps, use the accumulated gradients to update the variables.\n",
+ "\n",
+ "In automatic mixed precision training, gradient accumulation is also supported, and the usage is as follows:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 31,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,\n",
+ " [1.25853443])\n",
+ "\n",
+ "AMP time:\n",
+ "total time = 1.034 sec\n"
+ ]
+ }
+ ],
+ "source": [
+ "model = SimpleNet(input_size, output_size) # define model\n",
+ "\n",
+ "optimizer = paddle.optimizer.SGD(learning_rate=0.0001, parameters=model.parameters()) # define optimizer\n",
+ "\n",
+ "accumulate_batchs_num = 10 # the batch numbers of gradients accumulation\n",
+ "\n",
+ "# define GradScaler\n",
+ "scaler = paddle.amp.GradScaler(init_loss_scaling=1024)\n",
+ "\n",
+ "start_timer() # get start time\n",
+ "\n",
+ "for epoch in range(epochs):\n",
+ " datas = zip(train_data, labels)\n",
+ " for i, (data, label) in enumerate(datas):\n",
+ "\n",
+ " # create AMP context environment\n",
+ " with paddle.amp.auto_cast():\n",
+ " output = model(data)\n",
+ " loss = mse(output, label)\n",
+ "\n",
+ " # use GradScaler complete the loss scaling\n",
+ " scaled = scaler.scale(loss)\n",
+ " scaled.backward()\n",
+ "\n",
+ " # when the accumulated batch is accumulate_batchs_num, update the model parameters\n",
+ " if (i + 1) % accumulate_batchs_num == 0:\n",
+ "\n",
+ " # update parameters\n",
+ " scaler.minimize(optimizer, scaled)\n",
+ " optimizer.clear_grad()\n",
+ "\n",
+ "print(loss)\n",
+ "end_timer_and_print(\"AMP time:\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 5. Conclusion\n",
+ "\n",
+ "As can be seen from the above example, using the automatic mixed precision training, in O1 mode the total time is about 1.299s, in O2 mode the total time is about 0.888s, while the ordinary training method takes 2.935s, and the training speed is increased by about 2.4 times in O1 mode and 2.4 times in O2 mode. For more examples of using mixed precision training, please refer to paddlepaddle's models: [paddlepaddle/models](https://github.com/PaddlePaddle/models)."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "py35-paddle1.2.0"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.4"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}
diff --git a/docs/guides/01_paddle2.0_introduction/basic_concept/amp_en.md b/docs/guides/01_paddle2.0_introduction/basic_concept/amp_en.md
index ee31dc70ba1..6c5f15edfae 100644
--- a/docs/guides/01_paddle2.0_introduction/basic_concept/amp_en.md
+++ b/docs/guides/01_paddle2.0_introduction/basic_concept/amp_en.md
@@ -1,6 +1,6 @@
# Automatic Mixed Precision Training
-In general, the datatype of training deep learning models is single-precision floating-point format(also called FP32). In 2018, Baidu and NVIDIA jointly published the paper: [MIXED PRECISION TRAINING](https://arxiv.org/pdf/1710.03740.pdf), which proposed mixed precision training. During the process of training, some operators use FP32 and other operators use half precision(also called FP16) in the same time. Its purpose is to speed up training, while compared with the FP32 training model, the same accuracy is maintained. This tutorial will introduce how to use automatic mixed precision training with PaddlePaddle.
+In general, the datatype of training deep learning models is single-precision floating-point format(also called FP32). In 2018, Baidu and NVIDIA jointly published the paper: [MIXED PRECISION TRAINING](https://arxiv.org/pdf/1710.03740.pdf), which proposed mixed precision training. During the process of training, some operators use FP32 and other operators use half precision(also called FP16) in the same time. Its purpose is to speed up training, while compared with the FP32 training model, the same accuracy is maintained. This tutorial will introduce how to use automatic mixed precision training with PaddlePaddle.
## 1. Half Precision (FP16)
@@ -55,6 +55,7 @@ import paddle.nn as nn
class SimpleNet(nn.Layer):
def __init__(self, input_size, output_size):
+
super(SimpleNet, self).__init__()
self.linear1 = nn.Linear(input_size, output_size)
self.relu1 = nn.ReLU()
@@ -118,19 +119,22 @@ end_timer_and_print("Default time:") # print massage and total time
```
Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
- [1.25010288])
-
+ [1.24072289])
+
Default time:
- total time = 2.943 sec
+ total time = 2.935 sec
### 3.4 Training with AMP
-Using automatic mixed precision training with PaddlePaddle requires three steps:
+Using automatic mixed precision training with PaddlePaddle requires four steps:
+
+- Step1: Define ``GradScaler``, which is used to scale the ``loss`` to avoid underflow
+- Step2: Use ``decorate``, to do nothing in level='O1' mode without using this api, and in level='O2' mode to convert network parameters from FP32 to FP16
+- Step3: Use ``auto_cast`` to create an AMP context, in which the input datatype(FP16 or FP32) of each oprator will be automatically determined
+- Step4: Use ``GradScaler`` defined in Step1 to complete the scaling of ``loss``, and use the scaled ``loss`` for backpropagation to complete the training
-- Step1: Define ``GradScaler``, which is used to scale the ``loss`` and ``gradients``to avoid underflow
-- Step2: Use ``auto_cast`` to create an AMP context, in which the input datatype(FP16 or FP32) of each oprator will be automatically determined
-- Step3: Use ``GradScaler`` defined in Step1 to complete the scaling of ``loss``, and use the scaled ``loss`` for backpropagation to complete the training
+In level=’O1‘ mode:
```python
@@ -161,14 +165,64 @@ for epoch in range(epochs):
optimizer.clear_grad()
print(loss)
-end_timer_and_print("AMP time:")
+end_timer_and_print("AMP time in O1 mode:")
```
Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
- [1.23644269])
+ [1.24848151])
+
+ AMP time in O1 mode:
+ total time = 1.299 sec
- AMP time:
- total time = 1.222 sec
+
+In level='O2' mode:
+
+
+```python
+model = SimpleNet(input_size, output_size) # define model
+
+optimizer = paddle.optimizer.SGD(learning_rate=0.0001, parameters=model.parameters()) # define optimizer
+
+# Step1:define GradScaler
+scaler = paddle.amp.GradScaler(init_loss_scaling=1024)
+
+# Step2:in level='O2' mode, convert network parameters from FP32 to FP16
+model, optimizer = paddle.amp.decorate(models=model, optimizers=optimizer, level='O2', master_weight=None, save_dtype=None)
+
+start_timer() # get start time
+
+for epoch in range(epochs):
+ datas = zip(train_data, labels)
+ for i, (data, label) in enumerate(datas):
+
+ # Step3:create AMP context environment
+ with paddle.amp.auto_cast(enable=True, custom_white_list=None, custom_black_list=None, level='O2'):
+ output = model(data)
+ loss = mse(output, label)
+
+ # Step4:use GradScaler complete the loss scaling
+ scaled = scaler.scale(loss)
+ scaled.backward()
+
+ # update parameters
+ scaler.minimize(optimizer, scaled)
+ optimizer.clear_grad()
+
+print(loss)
+end_timer_and_print("AMP time in O2 mode:")
+```
+
+ in ParamBase copy_to func
+ in ParamBase copy_to func
+ in ParamBase copy_to func
+ in ParamBase copy_to func
+ in ParamBase copy_to func
+ in ParamBase copy_to func
+ Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
+ [1.25075114])
+
+ AMP time in O2 mode:
+ total time = 0.888 sec
## 4. Advanced Usage
@@ -204,7 +258,7 @@ for epoch in range(epochs):
scaled = scaler.scale(loss)
scaled.backward()
- # when the accumulated batch is accumulate_batchs_num, update the model parameters
+ # when the accumulated batch is accumulate_batchs_num, update the model parameters
if (i + 1) % accumulate_batchs_num == 0:
# update parameters
@@ -216,12 +270,12 @@ end_timer_and_print("AMP time:")
```
Tensor(shape=[1], dtype=float32, place=CUDAPlace(0), stop_gradient=False,
- [1.25127280])
-
+ [1.25853443])
+
AMP time:
- total time = 1.006 sec
+ total time = 1.034 sec
## 5. Conclusion
-As can be seen from the above example, using the automatic mixed precision training, the total time is about 1.222s, while the ordinary training method takes 2.943s, and the training speed is increased by about 2.4 times. For more examples of using mixed precision training, please refer to paddlepaddle's models: [paddlepaddle/models](https://github.com/PaddlePaddle/models).
+As can be seen from the above example, using the automatic mixed precision training, in O1 mode the total time is about 1.299s, in O2 mode the total time is about 0.888s, while the ordinary training method takes 2.935s, and the training speed is increased by about 2.4 times in O1 mode and 2.4 times in O2 mode. For more examples of using mixed precision training, please refer to paddlepaddle's models: [paddlepaddle/models](https://github.com/PaddlePaddle/models).
diff --git a/docs/guides/01_paddle2.0_introduction/basic_concept/autograd_cn.rst b/docs/guides/01_paddle2.0_introduction/basic_concept/autograd_cn.rst
index 3951f03c09d..fcf36e1d774 100644
--- a/docs/guides/01_paddle2.0_introduction/basic_concept/autograd_cn.rst
+++ b/docs/guides/01_paddle2.0_introduction/basic_concept/autograd_cn.rst
@@ -35,7 +35,7 @@ PaddlePaddle的神经网络核心是自动微分,本篇文章主要为你介
.. parsed-literal::
- 2.1.1
+ 2.2.0
本案例首先定义网络。因为本示例着重展示如何使用飞桨进行自动微分,故组网部分不过多展开,直接使用高层API中封装好的模型\ ``vgg11``\ 。
@@ -291,4 +291,4 @@ PaddlePaddle的神经网络核心是自动微分,本篇文章主要为你介
五、总结
------------------------
-本文章主要介绍了如何使用飞桨的自动微分,以及飞桨的自动微分机制。
+本文章主要介绍了如何使用飞桨的自动微分,以及飞桨的自动微分机制。
\ No newline at end of file
diff --git a/docs/guides/01_paddle2.0_introduction/basic_concept/gradient_clip_cn.rst b/docs/guides/01_paddle2.0_introduction/basic_concept/gradient_clip_cn.rst
index 5f32441212d..7d5cd89b959 100644
--- a/docs/guides/01_paddle2.0_introduction/basic_concept/gradient_clip_cn.rst
+++ b/docs/guides/01_paddle2.0_introduction/basic_concept/gradient_clip_cn.rst
@@ -20,6 +20,8 @@ Paddle提供了三种梯度裁剪方式:
.. code:: ipython3
+ import paddle
+
linear = paddle.nn.Linear(10, 10)
clip = paddle.nn.ClipGradByValue(min=-1, max=1)
sdg = paddle.optimizer.SGD(learning_rate=0.1, parameters=linear.parameters(), grad_clip=clip)
diff --git a/docs/guides/01_paddle2.0_introduction/basic_concept/gradient_clip_en.rst b/docs/guides/01_paddle2.0_introduction/basic_concept/gradient_clip_en.rst
index b6d58570b4f..31fd73f8b11 100644
--- a/docs/guides/01_paddle2.0_introduction/basic_concept/gradient_clip_en.rst
+++ b/docs/guides/01_paddle2.0_introduction/basic_concept/gradient_clip_en.rst
@@ -20,6 +20,8 @@ By default, Gradients of all parameters in SGD optimizer will be clipped:
.. code:: ipython3
+ import paddle
+
linear = paddle.nn.Linear(10, 10)
clip = paddle.nn.ClipGradByValue(min=-1, max=1)
sdg = paddle.optimizer.SGD(learning_rate=0.1, parameters=linear.parameters(), grad_clip=clip)
diff --git a/docs/guides/01_paddle2.0_introduction/basic_concept/tensor_introduction_cn.md b/docs/guides/01_paddle2.0_introduction/basic_concept/tensor_introduction_cn.md
index 3eb03db37b8..00efa373a39 100644
--- a/docs/guides/01_paddle2.0_introduction/basic_concept/tensor_introduction_cn.md
+++ b/docs/guides/01_paddle2.0_introduction/basic_concept/tensor_introduction_cn.md
@@ -81,8 +81,8 @@ array([[1., 2., 3.],
**Tensor**不仅支持 floats、ints 类型数据,也支持 complex numbers数据,如果输入为复数数据,则**Tensor**的dtype为 ``complex64`` 或 ``complex128`` ,其每个元素均为1个复数:
```python
-ndim_2_tensor = paddle.to_tensor([[1.0, 2.0, 3.0],
- [4.0, 5.0, 6.0]])
+ndim_2_tensor = paddle.to_tensor([[(1+1j), (2+2j)],
+ [(3+3j), (4+4j)]])
print(ndim_2_tensor)
```
@@ -473,7 +473,6 @@ x.logical_not(y) #对两个bool型tensor逐元素进行逻辑非操
### 线性代数相关
```python
-x.cholesky() #矩阵的cholesky分解
x.t() #矩阵转置
x.transpose([1, 0]) #交换axis 0 与axis 1的顺序
x.norm('fro') #矩阵的Frobenius 范数
diff --git a/docs/guides/01_paddle2.0_introduction/basic_concept/tensor_introduction_en.md b/docs/guides/01_paddle2.0_introduction/basic_concept/tensor_introduction_en.md
index 9e44ad029a7..f9dfcde4c58 100644
--- a/docs/guides/01_paddle2.0_introduction/basic_concept/tensor_introduction_en.md
+++ b/docs/guides/01_paddle2.0_introduction/basic_concept/tensor_introduction_en.md
@@ -80,8 +80,8 @@ array([[1., 2., 3.],
**Tensor** supports not only floats and ints but also complex numbers data, If input complex number data, the dtype of **Tensor** is ``complex64`` or ``complex128`` :
```python
-ndim_2_tensor = paddle.to_tensor([[1.0, 2.0, 3.0],
- [4.0, 5.0, 6.0]])
+ndim_2_tensor = paddle.to_tensor([[(1+1j), (2+2j)],
+ [(3+3j), (4+4j)]])
print(ndim_2_tensor)
```
@@ -482,7 +482,6 @@ x.logical_not(y) #logic not operation for two bool tensor
### linear algebra operators
```python
-x.cholesky() #cholesky decomposition of a matrix
x.t() #matrix transpose
x.transpose([1, 0]) #swap axis 0 with axis 1
x.norm('fro') #Frobenius Norm of matrix
diff --git a/docs/guides/01_paddle2.0_introduction/load_old_format_model.rst b/docs/guides/01_paddle2.0_introduction/load_old_format_model_cn.rst
similarity index 100%
rename from docs/guides/01_paddle2.0_introduction/load_old_format_model.rst
rename to docs/guides/01_paddle2.0_introduction/load_old_format_model_cn.rst
diff --git a/docs/guides/01_paddle2.0_introduction/migration_cn.rst b/docs/guides/01_paddle2.0_introduction/migration_cn.rst
index f04a2ee8835..94f9e2ee60d 100644
--- a/docs/guides/01_paddle2.0_introduction/migration_cn.rst
+++ b/docs/guides/01_paddle2.0_introduction/migration_cn.rst
@@ -66,7 +66,7 @@ paddle_upgrade_tool 可以使用下面的方式,快速使用:
开始
^^^^
-在使用paddle_upgrade_tool前,需要确保已经安装了Paddle 2.0.0版本。
+在使用paddle_upgrade_tool前,需要确保已经安装了Paddle 2.0.0+版本。
.. code:: ipython3
diff --git a/docs/guides/01_paddle2.0_introduction/update_cn.md b/docs/guides/01_paddle2.0_introduction/update_cn.md
index 2e1c44ab4ac..7f367547d13 100644
--- a/docs/guides/01_paddle2.0_introduction/update_cn.md
+++ b/docs/guides/01_paddle2.0_introduction/update_cn.md
@@ -558,5 +558,5 @@ https://github.com/PaddlePaddle/paddle_upgrade_tool
### 2.0文档教程
以下提供了2.0版本的一些示例教程:
-你可以在官网[应用实践](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/tutorial/index_cn.html)栏目内进行在线浏览,也可以下载在这里提供的源代码:
-https://github.com/PaddlePaddle/book/tree/develop/paddle2.0_docs
+你可以在官网[应用实践](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/practices/index_cn.html)栏目内进行在线浏览,也可以下载在这里提供的源代码:
+https://github.com/PaddlePaddle/docs/tree/develop/docs/practices
diff --git a/docs/guides/02_paddle2.0_develop/05_train_eval_predict_cn.rst b/docs/guides/02_paddle2.0_develop/05_train_eval_predict_cn.rst
index 789be2a9394..3c2182c9b33 100644
--- a/docs/guides/02_paddle2.0_develop/05_train_eval_predict_cn.rst
+++ b/docs/guides/02_paddle2.0_develop/05_train_eval_predict_cn.rst
@@ -7,7 +7,7 @@
.. note::
- 高层API实现的模型训练与预测如\ ``Model.fit()、Model.evaluate()、Model.predict()``\ 都可以通过基础API实现,本文先介绍高层API的训练方式,然后会将高层API拆解为基础API的方式,方便对比学习。最后会补充介绍如何使用paddle inference进行预测。
+ 高层API实现的模型训练与预测如\ ``Model.fit()、Model.evaluate()、Model.predict()``\ 都可以通过基础API实现,本文先介绍高层API的训练方式,然后会将高层API拆解为基础API的方式,方便对比学习。
一、训练前准备
---------------------
@@ -137,11 +137,6 @@ numpy_ndarray_n是对应原始数据经过模型计算后得到的预测数据
除了通过第一部分的高层API实现模型的训练与预测,飞桨框架也同样支持通过基础API对模型进行训练与预测。简单来说,\ ``Model.prepare()、Model.fit()、Model.evaluate()、Model.predict()``\ 都是由基础API封装而来。下面通过拆解高层API到基础API的方式,来了解如何用基础API完成模型的训练与预测。
-
-.. note::
-
- 对于网络模型的创建你依旧可以选择Sequential组网方式,也可以采用SubClass组网方式,为方便后续使用paddle inference进行预测,我们使用SubClass组网方式创建网络,若后续使用paddle inference预测,需通过paddle.jit.save保存适用于预测部署的模型,并在forward函数前加@paddle.jit.to_static装饰器,将函数内的动态图API转化为静态图API。
-
.. code:: ipython3
# 定义网络结构( 采用SubClass 组网 )
@@ -153,9 +148,7 @@ numpy_ndarray_n是对应原始数据经过模型计算后得到的预测数据
self.linear_2 = paddle.nn.Linear(512, 10)
self.relu = paddle.nn.ReLU()
self.dropout = paddle.nn.Dropout(0.2)
-
- #后续若不使用paddle inferece,可对 @paddle.jit.to_static 进行注释
- @paddle.jit.to_static
+
def forward(self, inputs):
y = self.flatten(inputs)
y = self.linear_1(y)
@@ -214,9 +207,6 @@ numpy_ndarray_n是对应原始数据经过模型计算后得到的预测数据
# 梯度清零
optim.clear_grad()
- ##保存模型,会生成*.pdmodel、*.pdiparams、*.pdiparams.info三个模型文件
- path='./mnist/inference_model'
- paddle.jit.save(layer=mnist,path=path)
.. parsed-literal::
@@ -284,101 +274,3 @@ numpy_ndarray_n是对应原始数据经过模型计算后得到的预测数据
.. parsed-literal::
predict finished
-
-
-部署预测模型
-=====================
-其中预测方法除以上两种外,还可采用原生推理库paddle inference 进行推理部署,该方法支持TeansorRT加速,支持第三方框架模型,支持量化、裁剪后的模型,适合于工业部署或对推理性能、通用性有要求的用户。
-
-
-四、通过paddle inference实现预测
------------------------------------------
-
-paddle inference与model.predict()以及基础API的预测相比,可使用MKLDNN、CUDNN、TensorRT进行预测加速,同时支持用 X2Paddle 工具从第三方框架(TensorFlow、Pytorh 、 Caffe 等)产出的模型,可联动PaddleSlim,支持加载量化、裁剪和蒸馏后的模型部署。针对不同平台不同的应用场景进行了深度的适配优化,保证模型在服务器端即训即用,快速部署。在这里,我们只简单的展示如何用paddle inference实现该模型的部署预测。
-
-4.1 准备预测部署模型
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-要使用paddle inference预测需得到paddle预测格式的模型,所以你需要在训练过程中通过 paddle.jit.save(layer=mnist,path=path) 来保存模型,注意在训练时在forward函数前加@paddle.jit.to_static装饰器,将函数内的动态图API转化为静态图API。在第三章节基础API模型的训练中已加入相关配置。
-
-.. code:: ipython3
-
- #模型目录如下:
- mnist/
- ├── inference.pdmodel
- ├── inference.pdiparams.info
- └── inference.pdiparams
-4.2 准备预测部署程序
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-将以下代码保存为python_demo.py文件:
-
-.. code:: ipython3
-
- import argparse
- import numpy as np
- from skimage import transform,data
-
- # 引用 paddle inference 预测库
- import paddle.inference as paddle_infer
- from PIL import Image
-
- def main():
- args = parse_args()
-
- # 创建 config
- config = paddle_infer.Config(args.model_file, args.params_file)
-
- # 根据 config 创建 predictor
- predictor = paddle_infer.create_predictor(config)
-
- # 获取输入的名称
- input_names = predictor.get_input_names()
- input_handle = predictor.get_input_handle(input_names[0])
-
- # 设置输入,自定义一张输入照片,图片大小为28*28
- im=Image.open('./img3.png').convert('L')
- im=np.array(im).reshape(1,1,28,28).astype(np.float32)
- input_handle.copy_from_cpu(im)
-
- # 运行predictor
- predictor.run()
-
- # 获取输出
- output_names = predictor.get_output_names()
- output_handle = predictor.get_output_handle(output_names[0])
- output_data = output_handle.copy_to_cpu() # numpy.ndarray类型,是10个分类的概率
- print(output_data)
- print("Output data size is {}".format(output_data.size))
- print("Output data shape is {}".format(output_data.shape))
- pred=np.argmax(output_data) #选出概率最大的一个
- print("The predicted data is : {}".format(pred.item()))
-
- def parse_args():
- parser = argparse.ArgumentParser()
- parser.add_argument("--model_file", type=str, help="model filename")
- parser.add_argument("--params_file", type=str, help="parameter filename")
- parser.add_argument("--batch_size", type=int, default=1, help="batch size")
- return parser.parse_args()
-
- if __name__ == "__main__":
- main()
-
-
-4.3 执行预测程序
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. code:: ipython3
-
- python python_demo.py --model_file ./mnist/inference_model.pdmodel --params_file ./mnist/inference_model.pdiparams --batch_size 2
-
-.. parsed-literal::
-
- #输出如下
-
- [[-1347.5923 -1156.918 -774.73865 3387.0623 -1553.3696 107.96879
- -2631.2185 -701.50323 -1094.3896 206.71666]]
- Output data size is 10
- Output data shape is (1, 10)
- The predicted data is : 3
-
-详细教程可参照paddle inference文档:https://paddle-inference.readthedocs.io/en/latest/quick_start/python_demo.html
-
diff --git a/docs/guides/performance_improving/index_cn.rst b/docs/guides/performance_improving/index_cn.rst
index 241893eca6b..64faa2caf93 100644
--- a/docs/guides/performance_improving/index_cn.rst
+++ b/docs/guides/performance_improving/index_cn.rst
@@ -2,6 +2,11 @@
性能调优
########
+你可以通过以下内容,了解飞桨框架性能调优相关的内容:
+
+- `模型量化 <./quantization.html>`_ : 使用飞桨框架进行模型量化。
+
.. toctree::
- :maxdepth: 1
+ :hidden:
+ quantization.md
\ No newline at end of file
diff --git a/docs/install/docker/fromdocker.rst b/docs/install/docker/fromdocker.rst
index aa25d82d3d7..62905f664d7 100644
--- a/docs/install/docker/fromdocker.rst
+++ b/docs/install/docker/fromdocker.rst
@@ -5,5 +5,4 @@
.. toctree::
:maxdepth: 1
- linux-docker.md
macos-docker.md
diff --git a/docs/install/docker/fromdocker_en.rst b/docs/install/docker/fromdocker_en.rst
index c0b2b487411..af6a1a7fafe 100644
--- a/docs/install/docker/fromdocker_en.rst
+++ b/docs/install/docker/fromdocker_en.rst
@@ -5,5 +5,4 @@
.. toctree::
- linux-docker_en.md
macos-docker_en.md
diff --git a/docs/practices/cv/image_ocr.ipynb b/docs/practices/cv/image_ocr.ipynb
new file mode 100644
index 00000000000..d3b9c516c16
--- /dev/null
+++ b/docs/practices/cv/image_ocr.ipynb
@@ -0,0 +1,722 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "# 通过OCR实现验证码识别\n",
+ "\n",
+ "**作者:** [GT_老张](https://github.com/GT-ZhangAcer) \n",
+ "\n",
+ "**时间:** 2021.11\n",
+ "\n",
+ "**摘要:** 本篇将介绍如何通过飞桨实现简单的CRNN+CTC自定义数据集OCR识别模型,数据集采用[CaptchaDataset](https://github.com/GT-ZhangAcer/CaptchaDataset)中OCR部分的9453张图像,其中前8453张图像在本案例中作为训练集,后1000张则作为测试集。 \n",
+ "在更复杂的场景中推荐使用[PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)产出工业级模型,模型轻量且精度大幅提升。 \n",
+ "同样也可以在[PaddleHub](https://www.paddlepaddle.org.cn/hubdetail?name=chinese_ocr_db_crnn_mobile&en_category=TextRecognition)中快速使用PaddleOCR。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 一、环境配置\n",
+ "\n",
+ "本教程基于Paddle 2.2.0 编写,如果你的环境不是本版本,请先参考官网[安装](https://www.paddlepaddle.org.cn/install/quick) PaddlePaddle 2.2 。"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "2.2.0\n"
+ ]
+ }
+ ],
+ "source": [
+ "import paddle\n",
+ "print(paddle.__version__)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 二、自定义数据集读取器\n",
+ "\n",
+ "常见的开发任务中,我们并不一定会拿到标准的数据格式,好在我们可以通过自定义Reader的形式来随心所欲读取自己想要数据。 \n",
+ "\n",
+ "设计合理的Reader往往可以带来更好的性能,我们可以将读取标签文件列表、制作图像文件列表等必要操作在`__init__`特殊方法中实现。这样就可以在实例化`Reader`时装入内存,避免使用时频繁读取导致增加额外开销。同样我们可以在`__getitem__`特殊方法中实现如图像增强、归一化等个性操作,完成数据读取后即可释放该部分内存。 \n",
+ "需要我们注意的是,如果不能保证自己数据十分纯净,可以通过`try`和`expect`来捕获异常并指出该数据的位置。当然也可以制定一个策略,使其在发生数据读取异常后依旧可以正常进行训练。 "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 2.1 数据展示\n",
+ "\n",
+ "

\n",
+ "
\n",
+ "\n",
+ "点此[快速获取本节数据集](https://aistudio.baidu.com/aistudio/datasetdetail/57285),待数据集下载完毕后可使用`!unzip OCR_Dataset.zip -d data/`命令或熟悉的解压软件进行解压,待数据准备工作完成后修改本文“训练准备”中的`DATA_PATH = 解压后数据集路径`。"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# 下载数据集 \n",
+ "!wget -O OCR_Dataset.zip https://bj.bcebos.com/v1/ai-studio-online/c91f50ef72de43b090298a38281e9c59a2d741eadd334f1cba7c710c5496e342?responseContentDisposition=attachment%3B%20filename%3DOCR_Dataset.zip&authorization=bce-auth-v1%2F0ef6765c1e494918bc0d4c3ca3e5c6d1%2F2020-10-27T09%3A50%3A21Z%2F-1%2F%2Fddc4aebed803af6c57dac46abba42d207961b78e7bc81744e8388395979b66fa"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# 解压数据集\n",
+ "!unzip OCR_Dataset.zip -d data/"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "\n",
+ "import PIL.Image as Image\n",
+ "import numpy as np\n",
+ "from paddle.io import Dataset\n",
+ "\n",
+ "# 图片信息配置 - 通道数、高度、宽度\n",
+ "IMAGE_SHAPE_C = 3\n",
+ "IMAGE_SHAPE_H = 30\n",
+ "IMAGE_SHAPE_W = 70\n",
+ "# 数据集图片中标签长度最大值设置 - 因图片中均为4个字符,故该处填写为4即可\n",
+ "LABEL_MAX_LEN = 4\n",
+ "\n",
+ "\n",
+ "class Reader(Dataset):\n",
+ " def __init__(self, data_path: str, is_val: bool = False):\n",
+ " \"\"\"\n",
+ " 数据读取Reader\n",
+ " :param data_path: Dataset路径\n",
+ " :param is_val: 是否为验证集\n",
+ " \"\"\"\n",
+ " super().__init__()\n",
+ " self.data_path = data_path\n",
+ " # 读取Label字典\n",
+ " with open(os.path.join(self.data_path, \"label_dict.txt\"), \"r\", encoding=\"utf-8\") as f:\n",
+ " self.info = eval(f.read())\n",
+ " # 获取文件名列表\n",
+ " self.img_paths = [img_name for img_name in self.info]\n",
+ " # 将数据集后1024张图片设置为验证集,当is_val为真时img_path切换为后1024张\n",
+ " self.img_paths = self.img_paths[-1024:] if is_val else self.img_paths[:-1024]\n",
+ "\n",
+ " def __getitem__(self, index):\n",
+ " # 获取第index个文件的文件名以及其所在路径\n",
+ " file_name = self.img_paths[index]\n",
+ " file_path = os.path.join(self.data_path, file_name)\n",
+ " # 捕获异常 - 在发生异常时终止训练\n",
+ " try:\n",
+ " # 使用Pillow来读取图像数据\n",
+ " img = Image.open(file_path)\n",
+ " # 转为Numpy的array格式并整体除以255进行归一化\n",
+ " img = np.array(img, dtype=\"float32\").reshape((IMAGE_SHAPE_C, IMAGE_SHAPE_H, IMAGE_SHAPE_W)) / 255\n",
+ " except Exception as e:\n",
+ " raise Exception(file_name + \"\\t文件打开失败,请检查路径是否准确以及图像文件完整性,报错信息如下:\\n\" + str(e))\n",
+ " # 读取该图像文件对应的Label字符串,并进行处理\n",
+ " label = self.info[file_name]\n",
+ " label = list(label)\n",
+ " # 将label转化为Numpy的array格式\n",
+ " label = np.array(label, dtype=\"int32\")\n",
+ "\n",
+ " return img, label\n",
+ "\n",
+ " def __len__(self):\n",
+ " # 返回每个Epoch中图片数量\n",
+ " return len(self.img_paths)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 三、模型配置"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 3.1 定义模型结构以及模型输入\n",
+ "\n",
+ "模型方面使用的简单的CRNN-CTC结构,输入形为CHW的图像在经过CNN->Flatten->Linear->RNN->Linear后输出图像中每个位置所对应的字符概率。考虑到CTC解码器在面对图像中元素数量不一、相邻元素重复时会存在无法正确对齐等情况,故额外添加一个类别代表“分隔符”进行改善。\n",
+ "\n",
+ "CTC相关论文:[Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neu](http://people.idsia.ch/~santiago/papers/icml2006.pdf) \n",
+ "\n",
+ "\n",
+ "

\n",
+ "
\n",
+ "\n",
+ "网络部分,因本篇采用数据集较为简单且图像尺寸较小并不适合较深层次网络。若在对尺寸较大的图像进行模型构建,可以考虑使用更深层次网络/注意力机制来完成。当然也可以通过目标检测形式先检出文本位置,然后进行OCR部分模型构建。\n",
+ "\n",
+ "\n",
+ "

\n",
+ "
\n",
+ "\n",
+ "PaddleOCR效果图\n",
+ "
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "\n",
+ "# 分类数量设置 - 因数据集中共包含0~9共10种数字+分隔符,所以是11分类任务\n",
+ "CLASSIFY_NUM = 11\n",
+ "\n",
+ "# 定义输入层,shape中第0维使用-1则可以在预测时自由调节batch size\n",
+ "input_define = paddle.static.InputSpec(shape=[-1, IMAGE_SHAPE_C, IMAGE_SHAPE_H, IMAGE_SHAPE_W],\n",
+ " dtype=\"float32\",\n",
+ " name=\"img\")\n",
+ "\n",
+ "# 定义网络结构\n",
+ "class Net(paddle.nn.Layer):\n",
+ " def __init__(self, is_infer: bool = False):\n",
+ " super().__init__()\n",
+ " self.is_infer = is_infer\n",
+ "\n",
+ " # 定义一层3x3卷积+BatchNorm\n",
+ " self.conv1 = paddle.nn.Conv2D(in_channels=IMAGE_SHAPE_C,\n",
+ " out_channels=32,\n",
+ " kernel_size=3)\n",
+ " self.bn1 = paddle.nn.BatchNorm2D(32)\n",
+ " # 定义一层步长为2的3x3卷积进行下采样+BatchNorm\n",
+ " self.conv2 = paddle.nn.Conv2D(in_channels=32,\n",
+ " out_channels=64,\n",
+ " kernel_size=3,\n",
+ " stride=2)\n",
+ " self.bn2 = paddle.nn.BatchNorm2D(64)\n",
+ " # 定义一层1x1卷积压缩通道数,输出通道数设置为比LABEL_MAX_LEN稍大的定值可获取更优效果,当然也可设置为LABEL_MAX_LEN\n",
+ " self.conv3 = paddle.nn.Conv2D(in_channels=64,\n",
+ " out_channels=LABEL_MAX_LEN + 4,\n",
+ " kernel_size=1)\n",
+ " # 定义全连接层,压缩并提取特征(可选)\n",
+ " self.linear = paddle.nn.Linear(in_features=429,\n",
+ " out_features=128)\n",
+ " # 定义RNN层来更好提取序列特征,此处为双向LSTM输出为2 x hidden_size,可尝试换成GRU等RNN结构\n",
+ " self.lstm = paddle.nn.LSTM(input_size=128,\n",
+ " hidden_size=64,\n",
+ " direction=\"bidirectional\")\n",
+ " # 定义输出层,输出大小为分类数\n",
+ " self.linear2 = paddle.nn.Linear(in_features=64 * 2,\n",
+ " out_features=CLASSIFY_NUM)\n",
+ "\n",
+ " def forward(self, ipt):\n",
+ " # 卷积 + ReLU + BN\n",
+ " x = self.conv1(ipt)\n",
+ " x = paddle.nn.functional.relu(x)\n",
+ " x = self.bn1(x)\n",
+ " # 卷积 + ReLU + BN\n",
+ " x = self.conv2(x)\n",
+ " x = paddle.nn.functional.relu(x)\n",
+ " x = self.bn2(x)\n",
+ " # 卷积 + ReLU\n",
+ " x = self.conv3(x)\n",
+ " x = paddle.nn.functional.relu(x)\n",
+ " # 将3维特征转换为2维特征 - 此处可以使用reshape代替\n",
+ " x = paddle.tensor.flatten(x, 2)\n",
+ " # 全连接 + ReLU\n",
+ " x = self.linear(x)\n",
+ " x = paddle.nn.functional.relu(x)\n",
+ " # 双向LSTM - [0]代表取双向结果,[1][0]代表forward结果,[1][1]代表backward结果,详细说明可在官方文档中搜索'LSTM'\n",
+ " x = self.lstm(x)[0]\n",
+ " # 输出层 - Shape = (Batch Size, Max label len, Signal) \n",
+ " x = self.linear2(x)\n",
+ "\n",
+ " # 在计算损失时ctc-loss会自动进行softmax,所以在预测模式中需额外做softmax获取标签概率\n",
+ " if self.is_infer:\n",
+ " # 输出层 - Shape = (Batch Size, Max label len, Prob) \n",
+ " x = paddle.nn.functional.softmax(x)\n",
+ " # 转换为标签\n",
+ " x = paddle.argmax(x, axis=-1)\n",
+ " return x"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 四、训练准备"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 4.1 定义label输入以及超参数\n",
+ "监督训练需要定义label,预测则不需要该步骤。"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# 数据集路径设置\n",
+ "DATA_PATH = \"./data/OCR_Dataset\"\n",
+ "# 训练轮数\n",
+ "EPOCH = 10\n",
+ "# 每批次数据大小\n",
+ "BATCH_SIZE = 16\n",
+ "\n",
+ "label_define = paddle.static.InputSpec(shape=[-1, LABEL_MAX_LEN],\n",
+ " dtype=\"int32\",\n",
+ " name=\"label\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 4.2 定义CTC Loss\n",
+ "\n",
+ "了解CTC解码器效果后,我们需要在训练中让模型尽可能接近这种类型输出形式,那么我们需要定义一个CTC Loss来计算模型损失。不必担心,在飞桨框架中内置了多种Loss,无需手动复现即可完成损失计算。\n",
+ " \n",
+ "使用文档:[CTCLoss](https://www.paddlepaddle.org.cn/documentation/docs/zh/2.0-beta/api/paddle/nn/functional/loss/ctc_loss_cn.html#ctc-loss)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "class CTCLoss(paddle.nn.Layer):\n",
+ " def __init__(self):\n",
+ " \"\"\"\n",
+ " 定义CTCLoss\n",
+ " \"\"\"\n",
+ " super().__init__()\n",
+ "\n",
+ " def forward(self, ipt, label):\n",
+ " input_lengths = paddle.full(shape=[BATCH_SIZE],fill_value=LABEL_MAX_LEN + 4,dtype= \"int64\")\n",
+ " label_lengths = paddle.full(shape=[BATCH_SIZE],fill_value=LABEL_MAX_LEN,dtype= \"int64\")\n",
+ " # 按文档要求进行转换dim顺序\n",
+ " ipt = paddle.tensor.transpose(ipt, [1, 0, 2])\n",
+ " # 计算loss\n",
+ " loss = paddle.nn.functional.ctc_loss(ipt, label, input_lengths, label_lengths, blank=10)\n",
+ " return loss"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 4.3 实例化模型并配置优化策略"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# 实例化模型\n",
+ "model = paddle.Model(Net(), inputs=input_define, labels=label_define)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# 定义优化器\n",
+ "optimizer = paddle.optimizer.Adam(learning_rate=0.0001, parameters=model.parameters())\n",
+ "\n",
+ "# 为模型配置运行环境并设置该优化策略\n",
+ "model.prepare(optimizer=optimizer,\n",
+ " loss=CTCLoss())"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 五、开始训练\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The loss value printed in the log is the current step, and the metric is the average value of previous steps.\n",
+ "Epoch 1/10\n",
+ "step 526/526 [==============================] - loss: 0.2182 - 13ms/step \n",
+ "save checkpoint at /home/aistudio/output/0\n",
+ "Eval begin...\n",
+ "step 64/64 [==============================] - loss: 0.1953 - 6ms/step \n",
+ "Eval samples: 1024\n",
+ "Epoch 2/10\n",
+ "step 526/526 [==============================] - loss: 0.1394 - 10ms/step \n",
+ "save checkpoint at /home/aistudio/output/1\n",
+ "Eval begin...\n",
+ "step 64/64 [==============================] - loss: 0.0416 - 5ms/step \n",
+ "Eval samples: 1024\n",
+ "Epoch 3/10\n",
+ "step 526/526 [==============================] - loss: 0.0296 - 9ms/step \n",
+ "save checkpoint at /home/aistudio/output/2\n",
+ "Eval begin...\n",
+ "step 64/64 [==============================] - loss: 0.0327 - 6ms/step \n",
+ "Eval samples: 1024\n",
+ "Epoch 4/10\n",
+ "step 526/526 [==============================] - loss: 0.0150 - 9ms/step \n",
+ "save checkpoint at /home/aistudio/output/3\n",
+ "Eval begin...\n",
+ "step 64/64 [==============================] - loss: 0.0228 - 5ms/step \n",
+ "Eval samples: 1024\n",
+ "Epoch 5/10\n",
+ "step 526/526 [==============================] - loss: 0.0102 - 9ms/step \n",
+ "save checkpoint at /home/aistudio/output/4\n",
+ "Eval begin...\n",
+ "step 64/64 [==============================] - loss: 0.0161 - 6ms/step \n",
+ "Eval samples: 1024\n",
+ "Epoch 6/10\n",
+ "step 526/526 [==============================] - loss: 0.1300 - 10ms/step \n",
+ "save checkpoint at /home/aistudio/output/5\n",
+ "Eval begin...\n",
+ "step 64/64 [==============================] - loss: 0.0164 - 5ms/step \n",
+ "Eval samples: 1024\n",
+ "Epoch 7/10\n",
+ "step 526/526 [==============================] - loss: 0.0199 - 9ms/step \n",
+ "save checkpoint at /home/aistudio/output/6\n",
+ "Eval begin...\n",
+ "step 64/64 [==============================] - loss: 0.0121 - 5ms/step \n",
+ "Eval samples: 1024\n",
+ "Epoch 8/10\n",
+ "step 526/526 [==============================] - loss: 0.0060 - 9ms/step \n",
+ "save checkpoint at /home/aistudio/output/7\n",
+ "Eval begin...\n",
+ "step 64/64 [==============================] - loss: 0.0133 - 5ms/step \n",
+ "Eval samples: 1024\n",
+ "Epoch 9/10\n",
+ "step 526/526 [==============================] - loss: 0.0084 - 11ms/step \n",
+ "save checkpoint at /home/aistudio/output/8\n",
+ "Eval begin...\n",
+ "step 64/64 [==============================] - loss: 0.0098 - 5ms/step \n",
+ "Eval samples: 1024\n",
+ "Epoch 10/10\n",
+ "step 526/526 [==============================] - loss: 0.0100 - 9ms/step \n",
+ "save checkpoint at /home/aistudio/output/9\n",
+ "Eval begin...\n",
+ "step 64/64 [==============================] - loss: 0.0109 - 10ms/step \n",
+ "Eval samples: 1024\n",
+ "save checkpoint at /home/aistudio/output/final\n"
+ ]
+ }
+ ],
+ "source": [
+ "# 执行训练\n",
+ "model.fit(train_data=Reader(DATA_PATH),\n",
+ " eval_data=Reader(DATA_PATH, is_val=True),\n",
+ " batch_size=BATCH_SIZE,\n",
+ " epochs=EPOCH,\n",
+ " save_dir=\"output/\",\n",
+ " save_freq=1,\n",
+ " verbose=1,\n",
+ " drop_last=True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 六、预测前准备"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 6.1 像定义训练Reader一样定义预测Reader"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# 与训练近似,但不包含Label\n",
+ "class InferReader(Dataset):\n",
+ " def __init__(self, dir_path=None, img_path=None):\n",
+ " \"\"\"\n",
+ " 数据读取Reader(预测)\n",
+ " :param dir_path: 预测对应文件夹(二选一)\n",
+ " :param img_path: 预测单张图片(二选一)\n",
+ " \"\"\"\n",
+ " super().__init__()\n",
+ " if dir_path:\n",
+ " # 获取文件夹中所有图片路径\n",
+ " self.img_names = [i for i in os.listdir(dir_path) if os.path.splitext(i)[1] == \".jpg\"]\n",
+ " self.img_paths = [os.path.join(dir_path, i) for i in self.img_names]\n",
+ " elif img_path:\n",
+ " self.img_names = [os.path.split(img_path)[1]]\n",
+ " self.img_paths = [img_path]\n",
+ " else:\n",
+ " raise Exception(\"请指定需要预测的文件夹或对应图片路径\")\n",
+ "\n",
+ " def get_names(self):\n",
+ " \"\"\"\n",
+ " 获取预测文件名顺序 \n",
+ " \"\"\"\n",
+ " return self.img_names\n",
+ "\n",
+ " def __getitem__(self, index):\n",
+ " # 获取图像路径\n",
+ " file_path = self.img_paths[index]\n",
+ " # 使用Pillow来读取图像数据并转成Numpy格式\n",
+ " img = Image.open(file_path)\n",
+ " img = np.array(img, dtype=\"float32\").reshape((IMAGE_SHAPE_C, IMAGE_SHAPE_H, IMAGE_SHAPE_W)) / 255\n",
+ " return img\n",
+ "\n",
+ " def __len__(self):\n",
+ " return len(self.img_paths)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 6.2 参数设置"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "# 待预测目录 - 可在测试数据集中挑出\\b3张图像放在该目录中进行推理\n",
+ "INFER_DATA_PATH = \"./sample_img\"\n",
+ "# 训练后存档点路径 - final 代表最终训练所得模型\n",
+ "CHECKPOINT_PATH = \"./output/final.pdparams\"\n",
+ "# 每批次处理数量\n",
+ "BATCH_SIZE = 32"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "### 6.3 展示待预测数据"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAABmCAYAAADIx5U3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJztfXeYZVWV/Tr35Vf1XuWuqk50NxlBxcCgMqIYGJURTCMiyQQKJgyjjAkZTKCiSMuYEBQBRWRARQX0pyM6oyMKAhI7h6quXPVyuvf3x97n7v2qHtVVDNCNddb39dev7rnh3HvPOfectdfe2wRBAAcHBwcHBweHpQpvT1fAwcHBwcHBwWFPwk2GHBwcHBwcHJY03GTIwcHBwcHBYUnDTYYcHBwcHBwcljTcZMjBwcHBwcFhScNNhhwcHBwcHByWNNxkyMHBwcHBwWFJw02GHBwcHBwcHJY03GTIwcHBwcHBYUkjupidjTEBAES9SLit4TeojP/Wsyt/zvHyu8GBrwcHB8Jtw8PDAAAbFNtT+9ttEd7mLzJw9mLjbJvd7zLvcbPvfW+BfT+tnsdCn9HsZ9PqWQWz/m8qa3FAJJi7v8eVbfitygx8P0AQtDrbEwdjTGBgEDGqTwTUJ6Imyn/X9RH8f8B/SfWDlttml3mqzLYy2n/lipVh2bahrbNPEMLjunr86ALVWlu/S/+RTjWnnq3PYObsNR/MrP9bXWexbVVqMPdI+7yD3Wx7tLH67XhZ9+tNddFnNS2fkZlVJvBbjDCe58H3ffh7QZ8AgEhE2mqDO7HHHwFfZT6w3wW7ybQY9x0cNFTTCr8P85Ut5DuxqMkQHRBFT3tH+Pf0zDhVgP9OqzOW+Rtg65pMSC2ny7T1zDPeHG773Oc+R8dV6GOSisu5GhX6P8vbyhVVtoB6L3Zy0ooym/0059unusjrPVGwj1SPMfb56Wc0ewzS9x6Zta3Vc7DnqultfGDdm3u9dv6jpl5mpo0OmMo1muoJAOlkHPnynn/KBgZxJJGJtYfb8tU8AKA72QkAmCpNhWVe+JGlJxxDLCyr8dPS2xr8lGp89zEjnaISVPlcdM73v+tDYdl7P3U27xSePER7nOoar9J1GkE5LIu0mJBVQeU+6rNPFb5DP2wFEVVqBwMqq8/bC6UsNut/OYNcu6L2D2bNI3QN7Dns8b5q2fYMUT5C1y7CRzbUc6iHk0IzZ//Z0P2lsz0LAJjisbJ50KX3GueJcz3QbTrC9yBPwvBIX2oU59xPe1saM4XiPLV64hAB0NmeDv+emqY+kYgmAQCVWiksi/PtVfnlRtWLr9t1xGMwK13MDLHp0znPgf588/yFbNuj09ZHRmSeMv1sZveBRU/D53mXpkWZrVd7m2zL55tPlWmXL1I+56O+wPayqMlQLBrDQE8vbvjhdeG21SsHAQCVAg34nloFDw72AwDGp2ea/geAeHuGKt7VHW6rVKmDvOtdNJAHdekwU6NDAID+ThrIG3WZDQUebQsWZPVbnGXQBC1WYPMMg4ZfSd20ahWtrt1iW/AYWS/N3HrGGnTupskQX05v82fV31OFlsUxvGyLaibD7si0Xl1NYSr8KavaD7x6tukKNfN+xRQO7RrletG5upcNhmXdvcv3CvYtYiLIJDK46T9vDLetXEkMTaVI7XdqSiZDa/bZh8oq1H7rFfn4xZIJAMDMjPSTSJS6qBej/xOppOwfp4lRucyTGbUkGgaxrCe//mQAQDYqk7XxzSMAgO5kFwAg7SXCsnqVzhV4ii3i+Vc9wpNSVSYfAzuUyDBqfLuNdqpEpS3MHjR1+4rzci6uZr92YKzz6StqtK5H/KZ9oqph2HPYsqaB3Nga82TIlwMbvGNDtdFanX5XLfMXka92nN9LvU5t3K/LODg2Rs97+bI+AEAiJsNuuUSTl0xbak4dCjPUfmIJmQDXmF2KJegcM6VcWPaCY47G3oCIZ9CejuKXv7ot3LbfvvsDAMq8gCmVCmHZ8hX0XCanqc1WqzLux2L2GasXHvC2gJ+LfqnhF9Sf9T9g7HGYvQ8Aw+/LNPiUenSxv+eydrKHHrOjs+o137hfb1G25xFpWvI0o9WEp9VYPHe/hX3rzJwDZZ926iaYnJwMtyWTNOlOJGmM27FjZ1j2gqNfjLHJfIvazYXTDDk4ODg4ODgsabjJkIODg4ODg8OSxqLMZLV6DUO7htDdI5ohL0I0XzJFFOT0uNBXY+N8HJtTMtlUWGaiRHtOju8Ktw0M9PJ1iKrPTY6HZStXkImkXiZaOGpkHleq7J5qXJgJTdDKXmnRykw2e/9Gq8vNa/56tGUauzccsaUDvnp+tuot62z3Ufdnxe8JNsvUqopSZXrZi1odhtg6Gkzxx5PUVrqy0o7qOaLPN2x4KNyWZq2F59H+k1PSHuJxg3Jtz6srG0EDuXIOq1evDrclEmR2MqzyX7FiRVgW8IMsFsk8ottxnU0rbW3KIM7myho7KoyOjoZF7e1EC2ey9Jw2b94clu07uAYAsO2BLQCAQ/Y7OCzryVI/K02QqaK9U67X3kbnyhXF/FKpkkmvYc1kRut1rI2V2oBRDcVjO5Thdu/Hquo4NEFbJaxpSuvHQscE60ChrSZWgDvreACo8c/5dG3RBI1LdWXaarDQwFMmrWSSTGEJNgEbZUq27d3jWsSTYnpcteKpAICd27cDAH7y85/Jcfy8DNsJX/ril4Rl9v1GVB1yRaL842wi7YhLH8rlCwvSTz7eaPgB8oUa1q6Tdu9x2ymW6PsQjckbmZwiM2KlSn2iq1vuKR8KQlRfny0S02/XbjKtTFuzn44u43cfWF3YQscWurY27fhmMW9h7+QjgnmU680l/LzCV9LiXYTQz8WbtU1/j2bbtOW4UonGEN1XoywlqFaprKND2s/o6MyC+8Te+SYcHBwcHBwcHJ4gLM61HuTunG6Tw8bGiNlZt5KEr4loV1hmV7+VMgni2jpELD2dJ1FpEBFx4BtOOhEAcNVV3wEAvOLYF4dlZRbVxXg1lsuLKCrTJqJbQIk6w1ovAjyznX+W+HgwQ63wf52rzq1ngl25WglJW3sCzHOvdZpzR9RKyM7SraC0Whc2oJpnr6QG7e8rAWoqzavugqyo7Uoc7JqcbhMRcLW68LXb4wkPHpKRZBMbsWHDBgBABzsJdHVJn7Bi6oCfXVS5TJZZcN3bv2zO/lZI2tXRGZZNTNEq2wqp16xZE5a19RLbcef//oXOo/rI6mXEYpV96p8z09KX4izMRVT6eCxGDcPzaDUWMXOFpB637UhDeT55dI6Iz2XRuW/MCqcVQQbWQzdtsyt9e4aoXoDOZpl0GIGgeVurJl7ndzE+OhFus+9stQpX8PCmjQCAW267leoQk3s9+mgSLzd8Ysh6e3vDsnvupHewetUqAMBrX/WqsKxconduRfO//fVvwrKdw8SEZjsz4bZYkt51rUHvQjOvfb09GFNi/T0FzwNSKWGDAGBsYgcAIM0eZh0dck9jY8R22mfXaAjT7LUcR2et9Rfof29mi4Kb2nEzw9HUTsJxW2/1msvMnBLFTmmhdrPvrh8oFngvgjfPI23ub7O+ly3KWtsrvFnXafGiW4Vg4W9BX1/fnHPlp2gc87z5fOEeGY4ZcnBwcHBwcFjSMPPZBmfDMyaIecCGh/4SbrvtFzcDAPITbPctyyozyytjL0armYaR1WaRV2PtHT3htvGpaf5Fc8mBZcIkHfUPzwYAVFnLsHKFrHSLM7Nm/ItmYBYbHGH3559Pc7QnYd2OW7pHtnDFbwWrc4gxe9BoyEot5jWfeNcu0YTd/9CDTdvqvhx3zMtfDgBYvXqfcNuGjZsBAGnWFhWKEk7h2c85CuUq0PD3bIA5z3hBHEk8fN/D4bZKiRiwBAdMsbZsAIhGeGXTTe1+bGwsLLPPUWtRJqZppT+4fDkAIKUCbNhj7RPIK7Y0xiyMYVamv1v6S4NjfMUjxLzpIWB0gs7Zv0JWXqM5Wrn73tygL1YrEWE3+mhDVmVR3uaxwKeUkBg4YQSGFpqehbjptupfwTwMp5nFEAESIiKepWeaTktcHMvUDI0MhdvaszSe9S4j5m7rts1h2e9+/3sAEjKhr1fGtWOOOQYAMDVOzJNtAwCQSdO1CwXSb3V2CvMXidH70UxJpUZtaSLHoUyicrO/+u3/w0Vf/AK2btu2x4MuRjxg5867w20BM1gRjppbV4yx/QZlsxyXS7FbsZiONmUxa/xtGu9tI4jO2deYAjSCecc7TT3ODRsRbgtasBCW/Qnd9bWm0t433bPvZ+epw57DfK71Cw8mtHAd7EI/l4FPz09/c2y4hoFBYl59FZG5s2MFGlhY0EXHDDk4ODg4ODgsabjJkIODg4ODg8OSxqLMZMaYIApg19B94bYc0+opdpVMJ8QUFo0xPc65M6byElm0bzmJOLfsFFfhtgxRhja6ZK9ysbzxRz8AALzl9FPpXONiXoi1oioXhVaRMR85gnRLN8LZe7aIXN0qInRrLMbG9lgy4q2u+8h1rqko4BbpBFH7Kf5fR9S1EZkD3mbdgwHg2pvZ3KpMPYPLyTV3204yVZx8mqRuOeCgg5ErBqg39g4z2QP3PBBui7Dy12P158SotNU40/69HHl9y5YtYdk+HJ1aR6BOtonpBgAeePDB8Hcbm20OPpjc5scmVOgBFqNWObVNVEWZrhTZDZ5zlCWSco0CuzfXPDFjRNIcJsGbm7jFiqOteSzekP4fr9O9erxPOS7935qoIi2aV427VV1bJWYdp6NMz44u3dA5EMNkfN6c4+y5CpwaQo+FiQTVXUf8rjbomRQ5erKnzF3tGXqGVvR/zz33hGV33/VXACKqfv7zjgrLKuwqbMseekiFlmCzka5XxI6zGTKvzRSlrRzxnCOQr5TR8Bt7tE9EIiZobwOGhjaF23xO+VLnvErT09NhWZpNhX29FD5ldETCs4gYVo1NZp5QKnOiU6tGFGHzW8txOPTJ5z+lHRvfnjOhdp91/qZzcj/xeHw0apy0v9lcFvjd2BshYvPd8CXc6cK9HnX4GIE/T+u1zhz6uzI9TdKZvmUkJRhV4+26tYc5M5mDg4ODg4ODw0KwKNd6z1BSPTsTA4Aory4DZkIaDZnBz3CQPC9Ks+hsRmbBI7tITGjUzD3OyS57uml1VVb5a2JxWiXt2EGrhnhUWIVYZB6xF6PltDCcxepZfXNe96DVkeFxMpeUHE12pvx/YYEebdat3c9tJedYqzzjra5rXU7n5qOybrLWzRcAChxGYXqK24hKKRxjxiTBLtf1qtThlceRgLqokq/aVbkNvnf55d8Myz72sY/h4ku+0aK+Tyw8E0F7vD0MIwEAjRo9j0yGmBsdBMw+axs0rL+/PyzLz9Azq9WkPXey88HWHRSwb9PGjVLG7t82yKNm2np6iWUdL09xneS5dvVSX8oxU1uPSlnPMhL+bp+Q/D6NKLs8s4Bat+2gYV1r6Z1qQtQPvY55jNBdaR5Xd7tfTTXnhmcZKPpbpUdDjM/lh/tKmZxj7pUs0ZVRIRss6sys6THIjnEpzixqlP+xdeyIcG7G/h4RQh/0muMBCAtUKUtiXJ/PuXM7uZ7vu3ZdWJYr0n415WhQY3bKhrCwgRkBoFgu7hX5+nwfKOSBnTulDVnysaOD6tvXJwLzCgv6dw3TN6GhKEFp0y0S1YUMUYs4CzYvXqAE2N4sCrEVQxTYQKEq36L9RvnqXJYZstv0uVgw7dkQAIplFV9yFlDPDhOwlyAIPQ2kfuJ8IM971u2g6Xs2hyVaYAiEeSwoU+w+r9t9s5t98ziYTBqUKgu7rmOGHBwcHBwcHJY03GTIwcHBwcHBYUljUWayABSTpK5yQnXYGDBTLKRWUYWTCaKyrJmsUhHKLZPmqLwqr1i9RtRkpUq0ZHtGTAj/+I/HAgB+/f8oQusLX/B8qdg8ZrLZ5HhzZE1Lx82NcuKHOWdUzhTTbB7z1dkl95k1G7SoUysqMayopgZbRC59RLSK0NJKEE7bKi2jANNxBq3qQGgou4SlUHflOQJyQmjJaJLeXTTBglplBo1Z6pnbQaUo5oIG53aycUgAoTunOMnd2884MyzrHdgfu89I9/jDDxrIVfKhqQoADItoSxxdOBlTz4dFt1ZAmk5Kvj5r+tBxhkoFMr+18X6veMUrwjIbj+W/fvtbAM35evwqnf+VrzyBNigR7uQ09dUkxwGbLIqYdWqCzD1+Uva3YuqAzRK6TzWsUwHT6UaZycHxPgKOu6LflxUv22amRZM17mdFpX2t8wH2KcfqckB0lpmsps5VnmUm02Y8+5hrZWt6Uv2G23ig4plYs1iMTXbaTFbjExfy9Cz7usVMZt9TMkG2IpvrDQC6ushcVInReDEyMiJ1iNDdxpIypkZ4fJ2YoL5Xqcm54tEk/BZODXsEAbCsT3KTReMsnM7R/UVV5gEE1LbLJXqe3Z0y7lerLWL1gIX4oXJej3f2vCx816Jn23paRYa2v40d9/WncdY5VZ29ULCtdrd1DWOuzRVX2xx+vtmtrnfPILSJqThNdosyIRreamZH5NZHziOqbhXbLpjnmfT0UH8ZHxdnESvNYXUCYmq8XQwcM+Tg4ODg4OCwpLHo3GQ0A5TDpiZpJdnfzSKmuszgi2VaJUVZ4RiPS/TcAovmzj7r/eG2SILKq6yYTSRkJl7K0XWuuPxbAIAf33hDWPZPL9oXANDJLJVeWRc5sqt1cx5QeZ82PkxiVC1iteLwZQO0qpmYlFVznTNZV3kFftrp4updrNDq8qYbf0wb1ErBCmLTnH9reFii2q5aTdfRUWbzeXaX5VlzKiXsQYSZBRt+IFBL6m9+kwTGSVYreiqxz4knnkTb+oiRs7mxACASp/dZzIswvi3Fqyhenb/tzDPCskw3h0Ao0P65soiHuzl31rf+g+oyrfI9pQ3N2JMsoE4pt3HrUJ9QEWdtri7LilimZa+EanPW7bNV9Fz7/sK8awr2/nReqyOPPJJPT+dfNiBtdXiUVtk2cvJNN90UlgU+sRE//yVlSI9FZYX8tGc+i+rC7vaRNqFgggQ13EJd3ukV37scALB9aBsAoF+JX5P8Tk977SkAgO4eyeVVmaBz2GDDVciKLR5hFoz/rqn2GDATUlDC0yxHdJ7YQkLjL3zm4rAM08QwZrqo7e2sS3iGMz/0PgDAsoFBroR0zMIk7ZeNNDuBAECe+0JK5Y5Lp1mozsPf1LSsTs877+MARMw+OiohQxIJar/pdhqfKmXp66kUsXMf+tdz+RoyRja481XKwvYkuA4BC29jCWlj1XoV/l6RsY8Q8aSNF1lgbtuh9nS2/hfpJL1jv67YUnbQ8ZTI/2XMjqba5q7lSwXa9rOf/I7PJe3eNzRmllgUb8MhAECtTm3Ijqeekefqc9gIowTUb3vTWQCAsRE6l45enkpSHaZy5PRw44+/G5ZVKrR/Ipnge5d2b8d5G8Vcu4/b8UJ/2yzj2NZGbebVr3613E+t2TIRVbkGb7iBvp12vGn1fYnw/hMTMn63pcjSo5kXq/8OquxAVZP7eeExlFu0M0vfi43K+eOAgw4CAFxz7dV0vZi8y44uGv+2bqYcj/r7nCvyu1MCavuYomxd0M+oXA4W7FTgmCEHBwcHBweHJY1HFXRxaJsEiosGNAMtTNIMsloSdqG7k1iITDv9PzYtK/tT3/IOAEBP/+pw29A4rx7iNFOdVKzMgQfsBwDYxQH4vnrppWHZX++gLPdHHHEEAGBmSoJ27bsvsUYTbGOMKMWD1VhEtZs+r1ymZmgGGonK6qZ/BeU+KRRplXLCa/8lLBsdo/u/9dZf0j0HEgytUrVB3WjW3KvyFo2NDwOQ1QoAdPfQCrKdXde1fXT9+vUAZKWQVAHzxlhb886z3w1A2dsBrOaM5nfvouf3P7ffHpbNTNPz6u+R0Ac51gP9/Jaf07lUZvoKa0dW7LcWALBl5/awLFxtMGPwtS/Ke0pzjqrlGZr5D2/ZFpYlOqiNRNS7sC7FDV6tRRRTuM/ap6IGwF9AMK3HExETCVJoxx//+Mdwm83Z1paid1MtiTYqzwynZdCKeXnvb33zWwAADzwoARyfcfgzAABXfOdKAM0B+E54NWU/t2zE8PBwWPaLX/+Qf9F6J5mWLOFXX/N9AEC2j973qGI4nv7cZwIAbrjlP8NtD22lQI/7H7I/XWdoR1hWzdH7Huwk5uU9p58dlvW0UTsvMXOTygorm99BrFaGdRxeUt57oYu2/WmXjDMpDgfw7YsuobpPS3v8yNupvUeYQTzv618Oy3b5tMr+2HmfoPOU5Lh2FhdlwjxZsoZcsZYYrofv/Wu4zYtwpnge8774xQvDsmKBWKYYB5pNt8vzHh0hjVYmQ208N6Nc65llv/BznwcAtCk3/wqvtsvVucxQlbVBgcpNdvBTD0IN/h7vE56JBDEkMbRL8vUVKzTuZDqoPcbjwkYUZugZ+FV6PtWSZk1p/5NOPj7c0t1H+1/53cvmXPu0U6j9capMXH2VsKUF/14AwAE8bu3YuTks61tGY67tQ/Go1K+rg9j7l77gNeG2oEFsTCJCdW5X77sWsqrU101UvomXrP93AMC6femcuYp8Ey2b08UhMzS7aAOx2sCsets///M/A2jOgfinP/8ZALB1M93jKaecgtm48cYbATQzUDZHnmEhnmapfbaMDG0Vy0ZXlsYQ+3k49sUvDcvSSXpGP77ppwCAzIC4wB/M99HOeQF/cdstYZnNtxfqRyPC2VS470VUwFNbfZ/1qTprff+ydS7oooODg4ODg4PDQuAmQw4ODg4ODg5LGos2k0UAbHlI0Z9MscfYZXC1Enh6LMz6/f/7LwDAf3zjO3Icu89//iv/EW5r66JjTznlrXR8VARrY7uIau5jE9PXvioU6fU/+BAA4A1veD0AIKqEwwGr82w+rEZVhGVW9GYFjgBQyBOFXalZ10flGp4kCvuoF5AwbO26A8Ky4RF6Dj+5mSjBZFGoxFicc7SxKSyTFdPWFIck0GK+DjZf3HvnHQCAG264PixrsKuvFXoPDoj76kknncz3RdSjjfwLiBDOX0amipQS8NZZAJ2KicjulJNPBABkOT/c8Se+Kix7aJjMF9/70XUAgHVPOSgsK7HZpFGi53zFV74elsXZdTbKubGWZcVcOMVu9hEl9KvZd2YFcnF5RqvXHbpXmMk84wUxJHHfX+9T26hKM1Nk5u1QFHoHiwkndhEFfuqpp4ZldW6buk9edx09Y0tf61xZ1jx29NFHAwB6+oTSLgTUriwVPpMTUXFvH/WzKRbAJ1V7vPTy9Xy8CKirEaLfS0z/a4Ei2JTTmyEKfGbnVFj0tje+DQCw72oyVRdKKrJwju61jcNF11VIhaE0nbO+j9zPeRd+BgAwfDeNPVefL6awFRFq7zkWnm725V7P+waZ1Ypc9sl3fSAsW9dBdc5w+3/wwfvDspUr6Bn1D6io+cNk1rVRFD7ykQ+HZXWODD0zQ/ff3p4Ny9Ip+r1ziGw3hxx6eFh26inkhLF6H3pGGzaIyLSvl/p2uSqmFDGT8TgWkbZy0NP2HjNZFO0YVmayBqi9J1JU71pNidWnqc8P9BwGAPCV9vekN5AJdCYv4+n3r/8qAKBSlxxUFokovbfXv4bMZdk2EfRffu3H+Nr0PP2GmK9GxujdPvVpT6W/h3aFZaefSufKJAbDbRd//tsAgGXd9D5e8Yp3hGWFHLWBjm5q7wMD0mcvvuQ8AECpzFKOtPpWcb+3Aurubml79lu1a5fUywqmrYD6+uvlO2HlClY4rR02jjrqqKbz//CHPwzLrJi63KA+Yb+7AJBpo3GsJyv9EvydfP2r6dv7t3tkHPzLHXfSuTg8SLcyuSFF3/bnPIv7ggpT8ctf/wqAOBVVVPR8pOh5OTOZg4ODg4ODg8NjjEW51ltYpgMA+lkUVWNX1B07RRTbxgKq5x75HADAs488Oiyr+jSj3rRDxJtegmaCn/3MRQCAd79X3O4POoRWDbuGSOAWjUvV38DisF/ddmvT9QAgbYWZvJoNPBEVW+H0rmERqnV0W0EYTTfTbbKqf+GLX0b31U5izl3jItROZ0n0NpWjWf2ypmBSLKrk2f3YmMy2Ozutu6LMSx++jzJe/+IX5BZdr2tXXJq5f+QjH6E67JK69y+jd7FzJz2jnh6Zifdxzqlxj2bk0xNS9/Y4vYtSSVbU1/+AGImpAon0Rgrynp59zPMAAK99Ez33U894i9wpB4Xr5qBzNe0eyu7U8bgNDyACecPLbZ0TyONgXSrsHfY+GBhEUFb5ppJ8L/Z9aybFiuFt8EUbVBEQV/y6EkImOfCkXTVqF167grS5efR1Mhli9PLM1K1bs39YtosD+3VxO85XRcQ9MUxt881nny7n6idmY/soCeVvvvlm2X+M2tHQEK3c+zMDYVkPu8RWOfiiZbcAoCNF9UuziHlGBQsssuB8ZFRWwX++n1acT1+1BgCQy8mqvsp9vMGC+7gK2TDFrriG3YF7B2R1PzlMdU9z33vqU58als3kiEnVK/EMs7dnnvEmAMDl3xZWu8Tskg0UWi7J/dicjJOcr0/nSurhTO07dpAofZ81kpusmKdzGEXee3MC2Ok+4eHR5zV87BDAwIfXxExbt+d6g96HZoYCvzlw4QP3ifNJboYDfkKsBFF2colYplg9E+Nzf+HcZPZ4AGjUaFujQc8slRb2zjrabNpEzgJW/AsA//E1YhfbYtJ2shx6hJsXrrlKLBVnnU2Wiolx+hY+9KCwfVl2IojaR6PS4lm2w44lRRWUdmqKnonOc9jG3ybLApVVXsd6nZ6vNZLospUryWnJBn5VQ3R4zYAF1AcccGBYNjlOjNfYmLjbZ5JUh7/9jVjVQw46NCxrMGvc3U33vG2TOERY56CuNuobQ+MqFyI7/mTYClRWDhvmceRvHDPk4ODg4ODgsKSx6KCLngcUlft8hW3lnTxTTnXJbLvCq6OREZr1zRSElUl3kT28u0tmuqOTtGre/0Aq04GVHn6IZtc2HcGXVcbyU0+lVe/hHEyut09ceIeH6No2iGA6JeccHbfpJCQw1xRrcVJpqteLXvxPYdntvyP36WP+ieowPCrMxgCnF0kKQYY1AAAgAElEQVTwcVFfVjd1XrFG2GUw1SarjgizJBPKvfmaH1wDACgW7SpKVplpXp1u3UGrjlRKBS5kZifL76CsjvNY+9TbTbPtbYqVGRyk57V1o9j4c+NUPrCSymLtsjIb5mCaM5xKZcXqVWHZgw9RoKxvf4NcwSe3iLv3zT8mW/C7TiZN2PSMsvnHw2Dvso3ZENNC1rY3ckQLgTePRM9q3SpqqWZdYr/1LQo2Wi4Kk1SwQSgt81IU5qXM7EoyQW1hZkLKsglqo+Osc1v/dQl/cP5HyQW9EpFV6SSHXnjKukMAAKnjRGP3058SSzQ1wmExYlLmcXiFzRwocXmfuAWDmYGAgyAm49K+Erzy/8KXvhBu2/8Q0qVNbGJWq0v0ZqkqM2r8d1K56Zc5qF1bmoa6M99xVlj2zU/R+StlqvtMTvpENELHFQuyrbuLWK/BFfT/qaeJu/IA6/ysRuMd73xXWDY8QuzSAfvTqrkxLu/CMoUDy2nM275NVsFdHZbZnbtmDTNR6G1z9tpzMIigocQ/jRq3pwi132yH6GGSMWqjI9yGLrnkq2FZNMJal4iMZTbYYM8yla+FMT5CrEU8xsc1RCvzTzxu3/47Cn+ydfvfwrKVK+m7sHYtvYct7JIOAKtWUFiXsWHV94pUn9wUvZuUCn45MUrjWluGxvlbbvxVWLbpYWJQeno5wHBB7su2BasBWrFC9KCbuT4vOuYl4bZMhliZNg6bkc1IChjLMll3fR2wtlyib3aCwxu8+lWvDcuuuYa+Pcks3c+994oGaDnrU+sqZMuJp74BANDVQf1xeLukk/n6ZTRmvfud7wQArDrg4LAM3Ed3cHtPdci48crjKIzCf950A9+LCoD5OAYVdcyQg4ODg4ODw5KGmww5ODg4ODg4LGksOmu97wM6KazVI5crRHF6KieKFZJ2dhItn2oXimuahVqZDjFpVTkD8CuOIxdxXwnwBpiaPpldvjlIJwDge1d9EgBwysl03M6dQjWnOCqtb/2zE8q9l8WO2ayY6qamicI+653k0vn968Tt8OGNm+g5cN6agcHlYVmJaf9Kw7rky73GOau7dbtNJGUOWqrQc+hdJtE5x9lc2NtLVLLniSnsXe8hytEKpweWi2DVRrq1AsFSUSjYlSvJxXTLZjKvDXZLCISR7WRK3He5mLtsZOxN95GgMO/JuXr2p/3e/34yBWzcJc+7f4CeSUhlKzr8bW8h89iXLqC8Uh9WAvkRZeKZAyuqnif78Z5CgAANNJpcV61rfd7mY1Ku9Tbr8tgIvb8mQTRT2VoIbQW8r3kNRb/V+cc6O4kWt+JqnQvtJz/7BQCgj91Z779X3MbbOWp5N4sXz//QeWGZz3L14oyI6btZjD21g9pXf0aE+aPbqK2m4mRmGNougv4qC4WfcsjTAQAzY+J2H2WjTr1o6XsZVJLsPr5rTEzHEY482830/7QScWYb1McDdtedGpe6p1Jkckil23kfEZKmOYt8o0wmbe36G/E4n6DK21ZiF/edQyQkX7tWzH6jYyQgt1GAr7zy22GZjVh/5jveCwDoU9nc27LUNnIsLu9UUeAh1ognHQI0oCKcwERtNG0a72oq910QtIVHAcD990tbTXE4k4RyMMnyMyuW2N0+0GU0rlnvap0HbqCfxiYrHM5kRa4wNU1tezpP9WrPSF8aGiaJRntKsiUUWdC8al8at4875kypQwfV2Wf39FJRzN5rVq3m+6exwfekfgc+5SkAgJ1sEtu+dWtYtnZ/koIMLJPvpQ2XYseLqnLisE46dlwaHBTx9zhHtrbRpf26NDR7/lyV+tDaNfuGZR5nAjAZed7j7EAR89gZaVTMZO//MDn5fPT9NM5fcKFEbB95mKLsd3DmAa0fmGFHgyq/u759JDzC2IwIrR9r7H1fFwcHBwcHBweHJxCLd603gFF5qgxTQwEHSKo3ZHZarbJwmN25q8qlNJGg2b1lIADgvR+goFgpFjv7yp1y+w6aJZ/G+Zu+8bWvhWUvecmxAIDvXX0tAKBWlTq87oQTqF7MWI2pPF/LmcXYpkSLq1aTa+uvf02BIleultXfCOdOKzCr5Sv3eS+e5PunewxUMDTrAm2D5BWLIspMJGkJMzIiz8HO2ANeDc/MCGvyifPoGXUy42KDLwKyijrrLGKPurqF8bIMQ1ea7rmkjsvEaIW8faOsRFYeuAYAkMxRnctJudeT+R2UYnSP/b3CMiVZlPdOFqr+8OtXhWWFaVptDPTSaqpekPckLpN6fh7M+n/vm7sbGMRMrMm13opo7XvUucm2bydWoYtzk11xxRVh2Ws5iJrOMD2wjJ6tXc2+9KWS+8cKLRPMCN2ggq7lc8RGTNSJjRnsldVVnoNB2txhQ6Oy2prI0aqxf40wlfkytb9lWdq2YfvmsKyXA2fazN7/yu0TAGpF6v/bt7ITgyf3FTXUTiLcrBoypCDgP9auXRtuu/2euwAA69aRCHlfVWY20Sq4xgEckylxiLB5nh7gYIbPWyMizhEOQTGQpeO0u34hT+NENiOM1fr1FPLDjk85LbiO2rbJ2b4nxTmAuz8uuuhzAIB4XJxMPvqxCwAA+QKvgvuF6fVVBvAnFwKQi7+MGTacRoWD4JaUE06jQm2ho4PeqWVPAaDErJrvy7OIsbNFudicmR0A0sz823xbmmXN52n8sUxqVLFNUe5Dk9PE+CxbLf0l0kPva8tDEmahp4MC7h57DOWnzCSFsRndRe2qq4fG1X95jWSTv/IqCmra1k71nFKOLNbJZRkHRdW5xu6/626qpxIT11jIbNkfTwUIXreWGB0ryh5WQST7epn9sXkSVXDHcWZc61E6dyMqzz3CuSU9I/04zvUp5Ik1PfBA6V+FUbr2BZ8lRujzF3w6LPvAeR8HAASc76y7X0w9pV00Xtrv5eROqbsORfBYY+/7ujg4ODg4ODg4PIFYvGt9BKhUxd4bYUbIutYHOrs5uw0aQzNyLyI22gavGnp7ZQV66XoKbnXiyRTGv39QNCw2A3uC9Tf1urL9t9G1jzvuODq3KvvBD34AADj5jW+k+mKuHkPbU63r/l/vppn4ZuViObhS7KcA0Nkts9nRcVpR2EBYcV9WlFu30jkOeyqtakdGpH4248g3vyVBu2ZydK5kkl6PTtXxlrcQK2PdKT/7WbHDWg3U15g1e9ObJBhiZydnzJ6kWXdfp6y+8ryKzabk/fi8Yimy7gVGNDGXXUqrmzd9kHRVk8rdu8bMmF3d2czKANAWJabqxH+h0O3XXnl1WHbsSSfikbH3ztkDBKgFtSbNkF2VWndW3ebsc6lVqA3YIJoAcNtttwEAjj322HDbpk2kU+vnAIbZrLAK9vxFfv5aM5T26F2WWZOzsl/0MF6F6rd9MzE23X0qXUgHHZcvy8rdrvA91qIN9qjAhRxeIhalNuqpIaVeoz7bwexRVDG25RmqVw8H/IR6DkEj33TvAPD0p5PuaPh+0rwN7RA2a22Er83pZBpKtzg9SczYCtb3Pfjgg2HZmjVrAAC1aWKG4yrVSSRqWW3pq2eccQYAIJuxTHde7U/t/v2sj9BhQSYnSUfR0UXb6orlqLJeo6OLmELLIgNA7NHFxN3jMDCIwkNFvW8/Su87FiN2wWo5AaDAjJ5lbCyDQxtp/4hKUVPgNDKZjrlBFwszOT4XPeOGohwT7Wxx4P6ZiMpxcQ7tEOdgvtPTokkr51mfOSABMQtT1Odu+jF9X176wteFZQP8PZmapvfe2yN91rK59Trdo84KX+e2MMJBUdtUCJbVq0lrpPuEzWBv2R+domLDBgpxYtu4dq23LLNlTfXztkxaknWFlrnR9Yuq8dhGAbHu76W8WDHst9BaAN7JLvYAcNEnSedrs2QMK/YnlqRztbdxiBiVjqYI+dY81th7vzIODg4ODg4ODk8A3GTIwcHBwcHBYUljUTxsBEBX3cOqiIiKKw2ioWfGieZKZlSEyDTRY3mfcwAp2i/gnESjYxvCbSuXU/6x71xGQsOz3v6JsKy3gyjKap6o7HNO+2JYdtUPJOcZABiVtfaNJxJ9WWbXVS8q9Gy6g2i8F/7TCeG2n95M+c0m2Ow1uFxMY0M7iY5MclTUeE3uNcLmiDYW+5ZVvqe1+9Lz2r6TcrM0Rchl13rr0g8AfX2WZiWq9pz3iQv6LhbnpVJEq5/7bx8Nyz7zGcrsHeEcYP/x9fVh2dlnU+bldMy6l0pusmiK6lOoCqU8EVLRHNU0KvcaKxFVeeX55CL/7nPeHZY1OOp4W5bOee45ks358m+Tu/HDU9RWhvuVO32MI6VWhBI1LCRuRGjOXlPZixvA4xiLdLFoYLokbuNtLGS3Qv72tLT7AouR0xypWUfXLrGJJKPyDwUsirR3PjEp1+nrX9Z03LHHSzu+6Rf/CwAo5ulZ7xiR41IcCT2SoP9jbWIe2jRBIvrYgFD7k5xRvsbi1E9/Rfpe+zKi+d/1VqLAy0Wh3JczzV1g2n96QO6rWmY6nk1vUWWyqPH7rqulWpWV1mUesb5/i4QYOOe1pwIAdnGbHY+JeaaYpjYdBFSvjk4xJU6VqT/3ePQuxnaJW7DNrVdvSF9N83u0GeNnip7an+71i5d8n849JWaWz32O+mWdIzLnclJ24YUkOL/oIhJnjysHD8RJJpDuUlHm2SQaidO1bU40AJR/cc8mrAdAuckaiCEVl/dtPGr3+RKZ5GOeiKv7OWPA9o3URiPqFnwbhV6JsQ2bxYxvx3mVm6zBOQ597l++tO1GhYTCdvytBfLtqXF4ha4sibjL0yLf8LjRVaIyXsXaqZ8UG38GAPzstx8Jy17y3PcBAAa7n0v3UJI296LnnwQAuP1PPwIADOUkwrM1W8UT1M4mJqTPxtlB5+CDRaD88MOS8wyQfGQA0NPT17St0VDjN5vTE/wc8gXlhJOm60yV6drtaTGhR9ikXZ5RMpkEmc6KUyy1yMi3t9Kgdp5g83OjLCbg976fvgs333ojACCoy3GTM2TG86wJvS7fJS9K5yiovG3dLAif4KwJEdWA2rMp5POy73xwzJCDg4ODg4PDksaiBdQxGEznZPVSqtGKK5lm98hOWcVM5mg22psh8WdJuR/HE7R/Ugm0imX6HYnSTPLqa68Jy844418BAB4L3bJqtW0z8lpXPJ3ZdudOElquW0tBq8YmRND7iuNeDgD4y5/vDrftGqXZbHs7rfQmJ4VBCV0ymXnS4mArbC3m6Hl0p2U1YAVx3T2cJTwvx1k3XRtAD1A5atpTTdcFREhr66WD9lkBte/PzXBuBb4Fzv9jWQUAGBqh4GWdSugXC+hcht3nh8dF4Jbl/QwLgz/3xc+HZR/9OK10R8ZtAEiZ1VsRn72HV79acuL8/Ec/BwAc88IXh9usuDjHrukl5aKezUYwnd87XI8DBOE9AcAMr1DsCqxXua5uYhdvP0IsQZvKlXf88ZSTR69QfvrTnwEQYe0rTzg+LLPntwHWIkpA/fxnUZ6+239FeZESgTyrDK8Ix3LUhqaHRIx84H7ERvzhYekTnWvp3ob4OiklAM6y4DIo0GqxOylOBYaFk708NowpcXAvv9vaGK22q2rlH2FX9/32Eff5//rLXwAA/7gvZZa3ATwB4NqvXgEAeOXJJMzvzcjY0Ma5+6ybckGxMo0qvYORUVoFL18hbu2WOQ1U5MNJDoa6zxpyu+6OyXu1ruJ2ta3F7M95znMAAHfdReEBiiqkhH2U93Nw04MOOkiuN8VhS8rKKYWfYTIi4QP2NkS9CDqTnaGwHwAiPrOkHIC00ZD3vXEj9Yk4yF09kZRnV61YF3kZA9vaeFzI2fAF0mMy7X1N+0ciUlav0buplGlbz3Lps5s2UR+oJKgtdCTFSaDKvGwkLnUwLLCfyef4b3mnv/sz9bnTT6BxcWxYvpf9y8hKMDlCz6ZzcO64b/Oq6Qz1mU5qa3/hfgAAa9as43vlPqi+EzYnmXUSOviQQ8IyG7rDlmmnDCuujmfZXV9FzpzhQJMJI+8nlrCBi+kc2pHEhhvJcwDXHuWolJvkHHL8zapB2vjAAPdDFttv2y7Z7lceMsjXkW+bFZV/5zsUxiUWlTrMzJQWbEFwzJCDg4ODg4PDkoabDDk4ODg4ODgsaSzKTOYDKKKBiici1zY2i0U5wuh0XsX4mKa5VpXjSOQLMveKMZ2cSgutnmf6ss4RLmuKor5/418BAKtXkbmrkBsKy7o62LTCwrqdw1J26D8cBQDY+jcSqgW+CLWWceTkZz7zWeG2TAfFRKmUiZ6NxkRw7UWozh1M9VZV7qAqU+42hs5NP/z3sKyvj6hbK4i2sZIAIMbPTQsnLQ1sqX1tqluxggTQlhrVcUls/IwYJ4+z+cEAyWPTzUL0ksoN1MaxZSYLEg01w4LQcTaJxjtUDCIWu9bYtJGIi8guX+bYOmzO7O+VPEw2JkUuV5hzXzbWhY6+bLd1s8AypoRwMzMN7B1GMsLUhNyLbeUxprutGQAAejnOUIKjMb/g+S8My/q4zNLXGjZuzS9+8Ytw2zHHHAMAWLmKzDbabLuS84cV2bypc9/5eXrPHTbmS7eY6u69i2j45z7vGeG2Ozb+DQDwo+vIbD2ocq295pWvonOwOHh6eHtYlqnTPXZ0UF2SUZXUsNpMXnsqFlPco99bN0pMlXUrKc5Kntvxjm3bwrKXv5zM3ddzBO5nnnhcWDa8nWIIWXPZofvuF5bZ+E82v59RhPrMNPWdQw4Rs1VHls4xNUV9Qou+bZ+28aO0aN6+45e+mOJHvec97wnL2vvpWT78IIl5/3qnmCf/+YQ3AaA8XxbWrGBj0BglFm0EDQR7gVuB7/solfLwfRkgk2w2tLFwYjEVZX2AxuGEIVNQVcWVCbg3KZ+Y0AQkcXW8OWUet6EA8j3yG1SHJOfRm5yQiPvLB8l8FdRp3KvmxVnDPvNqVfpXpTTDZdTH29Niatr8ADm5bNtKbS+ZkP6Saac+7hkrgZAx2t5PH+cHm56S6/2N496tWyexjiY5FpyVh9RUfC0rxl7G59qs4hNZ+UQ7m5P1OBzGHuJMEjMqR2Ga49BFG9LmpriOkTpt27hhc1jWsYYdrXi8L+XkfjJW7sHmaBOTcxo+/yjnADzw4APDsp3T1O+1CXH//cnJ6c1vfjMAkcQAwOjIJH50w4+xEDhmyMHBwcHBwWFJY9FZ6+sA2jplhVdlt7exMZr1VWsqQ+86Em0ND9Nqrq//gLAsx7l4JqeE2fBY+ORxFNCZkjAVBx1GYkqPXTIj7bICmpqk2atlUlYul+zC9/2BXIy7Omll3KdEc2ARX7ZdZpk+C7rtakDngpngmXghV9SH0zmyJIRLMitTVYxNWz9de2aaZrzxmHI/ZFGwjZALAEVedVghdHensGfDQ8R6WXGxzl9TYbf2FIvT41Gp+/IBYmiGdtCqtq6EzesOpJn1yBbJpzQ9whFSV1Ddq4GsOsZ4hu+xi7YWi04wm7N65Sq+Z1lZWPdOw2Leg/d/Sli24gS6/x/96EfhtreeSZmghzl7uadEo54HqNRwexQGQF+3RPTeNUQrwzaOsqvF8TV2L2WSEcmU9CXLhOncTD5HUy/z+7KrQAC45ZZbAIhAd2C5CBSHJ2hVNbCCVttW/AsA0YDbDEeLjylWpqOLVn9/uON34barf0LvZIQF1x/8tw+HZXGf1lMre4hdyW+TrPWDabp2fpRXf2m5L8Nt2wouAy10ZaH9cvVMTSetvEtT9GxXr5D+gp3UL1/NoQW+9fOfhkVdzAjZVffkLqlfwNTu8AS1e7sqBoBsltiDXbuEZbbPy44N8bjU2eYiS7NY3Kg8UTY6r422a92eAWCU8zcddRSFB9Hv97rrrqNtSlDcliGW4cCDaSxNt4uQdG9BAB/VoII2FTk/nqR2nyvQ89c5FbPszj0+TCLyWFJFOGarQkONP9M5YlN6llnWXtrv+C5qaw3QuBqN62RWVIdag9qXrxg3jpaARpXH6KgwPQ0bXqEhTFciSn0tkaS+VKuJBWHVSuoLWe5LpYKw8JNcd3D4h5oahwvc7scmeIxWrvKHPJUcB3JFFeE5S23BCpV1BPVpFnZbB5b2DrmfHWw5SSYTfB25L4+Z+VKOntXggDgVlAtUvynFJC3ro/IyR572IO0+t5Ouk2lr5zoJM5Tgd55mtrWsvpelEp3rC1+gcBOf+OR5chwzsNbSAQBxFlPbPqsdlK6+5ioUi8qEMw8cM+Tg4ODg4OCwpLHo5DcNA5QaMqsPOGdVRw/pAsplmaWXyrSiecfZFBgwEpHZ89EvJPt5Qs1m3/QmygA8xczLRE60E8PjZPNMpmlm+JVLvxyWpQJabXTzSrJcErZk1QpiiXxelQ3tkOzw115LWe5tZmQAiHHAq2SKZt2jozID7ewm++trX0OBHE1EHp/PrMe1136PzuM9FJZtY33DctYmjE/I6rTCQQZPPFFyc13CQe2sdkhnXraMUJgZXa0yLQMxOkrnt66XADA8TPe9YhVpJjxls79/A+mpzj1PAofZQHxfuowCN07lZXXjtbM+IrRRywqmq6+X60wrhPPPvyAsq5Vo/5UraVW/6T55Rk9ZQwE3CwVZ+WzZQi6VpSrnsVsmrF4kAtT2vDyC8vUBTVnrrYurXZXVS7LqifDKya7GtDtsg1lW/Qysfd9mu2/LCItp28LyAVqlRhUTWO2k824tcl6sTp13i845NkLnLClX+fRq6kM9DWEoCkkqf/+/ngsAyJWkLawYoHf56z/+NwDgoBVrwrKPfZmyVJ/2+jdQnYpKf8c6sxjrbgo1eUYN/v3hd58TbnvHhynw6GGD1KZ3bBa9x75Z6gMTvKJ87rOOCMt+fS8FxUswY3v+++ScZobao+2XNVUH65qt9ToTE7Zfrea/hUm17sDWFdm2XUBySNl3rUNE1LkR93T3cZnUweZJ1HWos/Cuj3PNlVX+LxMGu9izCIgbCsc2AAgiHFCUma2ocn/u7aT2m2KN3Xe++42w7MTXnwIA8H1pc51d1AfqdWs5MKqM2m+1Rtf2AulLN/zsUgBAobIZAJDtFBYjyu7iG7YSq3DGqSeHZX3dpEH51rc+FW5Lxlg/NkbniqncjTX+njR81kGm5Dl87wc0tsdSHHpEseo2e7zVIU2qjPbTzMZcc42Emzn11NPp7vn7oEMZfP7z5NYfMkMqV97atWRlqXDA35tukgCmQ2x5iDBrpIN4WpZfM5vfuPxbAIBXHUesbFta+rhlqmyQ0gHl3j/8EH1zkjyGjapv4p133wkAiLGm6ay3ShiNz62ngMyWnQUQ9o4c5zvMVWV+UirVQ63q7uCYIQcHBwcHB4clDTcZcnBwcHBwcFjSWFwEag+Ip4BqQ8xKltbtsDScEXq9wcmFCjkyCfT2CcV/7z0klnvwYTGVFJh+9yJ0/r/+7Y6wrM50Z5SjgHZ2S9W9SXbJZxFWrSLUsXVZj7Jbb/+AuHpbGjKuRItW3Fyt0PWSCblOe5rFZUUSwaWS4j67c5hMejGO2JxUwutahcMOsKu7zkeV5PwrfZzjCQBWLifx8QMPEJX4hc9LhOcPfvCDAMSlfsvmzWFZMU80YYLp5lPeKFSvpQqHhynSas0XQWJPLwlHragNABoeVfK1J1KU6HhW6M8vfeUSuv92uv93nvWusCzF9z2yi2jgFcsGVRnd67YNZELoyohgdcPDDwMATjrpjeG2P99FdOkhhx1Ox6ekDrXa3pGbzINBOprCjm3iUt7P7qwx9gfepcImrGWTiRdQmXaHtabPfVZK7r8dO8g9d82aNQCaQynMsOnSmtx0tO9KH52/0k59Y8qT6yTYjdX0k6gyFxUeeahA7+2CL18YbpuoUV9Y/93LAQCjKu8WpwzDun5qs9f/5MawrI3NyN4g9ftMRUWNz1FbLdU4z5Sn8gmxqDrZKwLqZxxIYvudPG709UjbKTE93tNN22Ya0mdTbEKYGiYaPqYi3rfFqD1u3EBjUH+/0P8dWamrxSfPo3ZvBZrxuFznM58lE4oVdq5cIePMjq3U56ybc70idfjmNyhf39QEjUVazJrtJtNbQ/XVGRbZW7NsUeUmC/YKx3oyWkVNgExWTDNTOTLTFyscqTsufTmXo20TE9TOupXYN8M5DmcK0kbr7FRQqdl7l7aTiJIZznB7z7SJOaVYvx8A0DAcnkFJJtrTZII/5NB/pOMDqUN+itrj644/N9yWylIbuPrar9B1MsIrnHDa26h+HFqgu0fqPl2iMT0eoX42MSrXsc4EHR0R/l++l3V2TFq/XvJN2sjzNvq9zjjw2c9+FoCY3LU5/kUvolAPPpvHzz1X7ssK+GMseh7aIdHpv3gRmfgmR8U8nOVMDRUexxJR7VxF2wxfe+Rv94ZlK9fQeDHBYSoynRJ+YIb7V22Gyr74ZcmFODZDZn/97Y2wyaydnQl8/9FxPI4ZcnBwcHBwcFjSMHrGuDt4xgQJD3jggVvDbfEozV7jMVqVTU/KyqZapVnmhRfRbHbjBlk9pzgo2eSMBJ1avopXZoZZjx0PhGWHP4PEVwccSKvmI4/6h7Ds6byCtKuyFcysAECJXc9LRTqnzTkGAMPsZrtunWSm3zVC57AC4PY2mbFu2kKr9Le9jVy+faXMauPzWnfYoC4CSjvj38lBpHrUqjbGTNSEcn0ucubvH/+YVtmaDfjwh8mt+cEHaYV8663yLoaGOMBeL7FMZ7JrOiDBsez9JNtlZVbgldZpZ7453FZjt/k6Z6uvR2X1lc7QqmGU2Z81q4XJGNpM93jl10hY55WErahN0wp+VT+tmutV5S7Lrpndyp16guscWIYhIquOw484AtP5OuoNf49qRqNeJMjG2nHrL24Jt9mVWpTbkBXvAtJGfWYAurKS3+rFL3oRgOZVnF1B3/Jzyt2mxdUnnECiRSugtIH4AODSX1I7HNpB76NfsSzVIq3Ee5fRtg1bJXv3VddRfp9qQv5LR2sAABmiSURBVNZJFY/qajiPns3HBAAlzmAdYTfgVd0icj/zVAoamE1Qe+kKhCUsbOd8fQExiVEliCx30D3/aWRzuC3ZS84B3/wCs5JTwk5/+kPkoDHMq8yrbrs5LPv5H28HAHz3yu9QHVR27GCEg092UPvSeZXsqntsXDLZX389PVPLrurgmKtW0Zhz3nnnAWhepdvciaeffjoA4MorvxuW2XEpzgyxZj/H85YpDzeFWesTHJJBC6ifceQzUIePINizqeuNMUEEUczMSFiC6TyNhz6oT/f2Cgu3fQu1p5WDTwMATI5pl3dqVyed/KpwW3cftc0rv3uZvWJYdtrJxFJbPe7V35WAe6bjjwCAFf00/u4YkXafjBFrV5mhsem0f/lMWBYDB7pV+f2KVXKKiSXofgoqYO2yPgqCWinTOP7Vr0sA3jWHURsr5oklSbc/OywrF2h/O1ZHFctSKdNYGVNjyaWXfhUAcNttt/H+wlR++9vEOFrh9FuVCLnAWep/+EP+VunxhpmhSWZudXtsYzbmk584L9z2iX+jXJRVDhny8pceG5bZoKY38Xesc0DyYR75dAoVMM599qHNEpi2UKD7r7MFyotKB/Bj9A5svwGAMrNS2Uwn34/s39e7FkGABfUJxww5ODg4ODg4LGksihmKGBO0R4AHH7g93Obzyq5c5GCFMWFe+gfIhe+uu4jh+cIXvhSWTfLsV9sKc3liico1mrle+Z2vh2X5Ms28DQc+++3tvw7LOqu0Wj78cNKWLF++MiwrcHoImwk4r1zEVzIj9OC994Xbutg104aBr9dlNZBM0GrvlFNOo3tQgcPsc/z+978PAEjFxZ3SMj02mF6xJIEIk0mbOkN0GKsPJFfOLawZuuyyy8IyO9O3wRa12/3ZZ9OqyLodToxLcCzLFlV5ZRlNiU4qxy6wqS6x8Z/0ltNpWzcxf1NFCWQV5VWsXYl88qPnhWVrOeDl+Fayxw90ClPgcQbqOms8EsoVvByn+2jS0LCLqc9LYx3Qc9+DDkcdC5vxP57wjAkSiOKyS+QdHX00BdCzgQW1K7UNQGbZvrhazVn33te97nXhtgJrRGJReuYN9Xws+2ADVWr3/pEoPePBftKdjI0Ii2HjG6ZSdL31X/9qWDY8RYxIvEMxG4auGWV3Wx34rTND/d1jZqgyJWXnnvMB+sHu45GqMFfZgO47zZoh/d4bnDF7xBP2p62PGLSxLcQ2fOtCGUsazE51csqRjTPipvuJL5IrbjsHIi3vkLK+ONU9iNDxk1PCzlo354hKd2FZvQsuOB9Ac9+zv21wx/EWfe9zn6MgcnaVDwgjNDpC19YZxEusAUu3CWNltRlxDsRYUu7rhz/7cJTqZTT8PcuWGmMCA2B0TMIf1Hy6v1Sa3rfVCQHA+q9cCQB411kU2iMWEbY0ymywFxUm4GWveD6dyzanQNb0JU759LOfUtBQvy7jnIkRmz7N+iUdbiLw6Tq5SWr3y7slQPCxLz6D6y7jlTH8fiPUP0sleQ83/5jY+qlJuse2rHxj69jB/xPjmCtIP7P92QZbTKdkPLaWimJRrmN1M5/+NIWw+N3vJFCqbUe2X1mmEwBuvOkGAMA+7GL/8INigbHhWdraqc3aMC2AjFVtKenHn/z4JwAA//5J6hMVNda96IUvbDpOa9/KnHLlLk7/oy1ExTKNIZ1dnBqlrlJO8avWLFiRx1LLouuwDV2dqxwz5ODg4ODg4OCwELjJkIODg4ODg8OSxqLMZDFjgs4IcNedQsdZk4k1IeVyKlKxzTXGWejrSnCc4XxgmnL/T6bvjj/+lbS/cuGPcnZ3GxlWi4qzETnHXMye78nfQcu5IG9jVi0wLfYJ5ptDcpZlVNW22c9YhcQME2z5c8vNAt5NE/v3yPdqEef8OtZ1HgA4AgKUthRV9pmu8TaVqBgNfia2ehFVhyjnEoqze3OiLnWIc2bjqG+PV7mdDLUbHVm0ynRxwFR5oISSBx92OMp1oOHvHWayu/8s2catqP1FLIjWpo86ZzUfGSH62eZwA4Aim8QintDxEX7WVjgdjchLSrLp11LG2kymTTgAYIK5bS7gF9jwpKxh7HuXbRWf6hzj0AtVZdLy+Dr2vS9TAvgH76Fs9/vtQ3T8mHIpT7GfRbJu6yfVKzMDXlC5v6p82zYKQEqqgBhf2za1ojw+1NjMFWlxXJp/16P83Fomu1vottnw5v4O27t6N+G2uX01z+LouArv0eDx2uYF1FnrD336oaj4NfjBnjWTeZ4JYlGDzVvE/BJP0jMrV8gckk6LCahconv64kUUeTriSdn730em1sBI267VyWSfTFGj0BYQG1UlYrL8v5h0DNjE5PH3whO5Agw3RJ+j/vsq51vAY5KRbw48MoF5bEKGL+Y4NLi/+1YCotqLR3UwhupQV+Y/tRNft0Ubatl2CBdeKOEw7PfxE58gM1ZT/jHuS21tdI82fAcAXH45hc8wHkepV+a1lctJSP6+974v3JabIvnKxRdfDAAYUFkCjns5RVDv7aMxwfelDtb8bPPXbdwkYvZ91pLUosrtv1qTd19p0PNOp+W9BsbODawpUfZfu+Zg1OuAv4DvhGOGHBwcHBwcHJY0Fs8MAbjjjt+H2xo8U7Or37oSSdmyeIKWag3FDFkBna9WsJ0dJJy74UbKlfLGk04Ny7Zto9mrzeGj3dpjEZm98m3Ncxe7mf/xbFvmkYtlhrgGyr1vzkqyifFpwQwtJh17y9XDI/0NJNmNXselssyAInFQs2yRZ9kDdcnw+dJGRSIg6tMqVhgizRpxGe8fUZXIt5NYNJmUFVmBA3oWmE2pN+RCBx12xF4joI7Dw6aHNoXbrADS5vzRLM0/PJvyZh188MEAgL/9TcT7vV3U/stKJJlJ0+rS9lMbpBMAKswE2bJAif077co2zLGlwO3L5/816+dzW2hiDvloy85q934r6J8a55ABFanDwUdQ+IsH//AH2rBCWKNEvfn/iKpgjfN7VVQXsm3T4/Ye1d2F/7f3UYt4c46z+ycVM5TgqtZars5b9ME5/bJVP51vvFgIMyTHR1kkrbPWW9f6UqXINZA6HPbMw/Ya13oDYNPme8JtmQ4bGI+Yl80qWGw8xm2cRcxdHZIp3YYqSCSlMbznvWcBANo4R2K1qrKSs0NPe5raWm5GWALPMjaWEYqIU0i4jRmipiHaskZeWW0roQm+sFloUD/2aiwED5JqR3ti7meJHZiLR8cM6XHG5sj76Ecp7MQFF0iOSGtxiXA/0TnNbKDHCXaEWM5sEADs2Ep11cFduzttPjV6F5Nj4oRwww1k6XnzmynEhmZ4rIXH5gPsVuFmCiygtrntEiooMnhciiiGPM9MkCW/bJ5QANh330McM+Tg4ODg4ODgsBC4yZCDg4ODg4PDksaicpMFABoAOto61TbipqzmMxIIZVmr2zgZRKEVFB3Xt4zoUh07Jp1iE1WN6LGxUckds2KQ6DrP0LnGxyUuj0nPoqvnNWM15inT5d6svxcJXyJ3zhVC6/oGLfYxLfazmHVvi2TE62xLCJSZ0Zoqm1K6sLnE6se9pssETf9rITQ4tg7YpKXzxDRsGV+6oUS9NkpvpSJ2DLY2Ic2xeXQEas8sTF/+eMPAIGZiYQ48AEhzzrZjX/ZSAEBD5cP65S9/CQBo5/xLmQ6JsxVLEZ2unQqqAT0PG/w2DhWpmUWSMRZQJ1Iqt1yenz//3WSY5XcZ8HtrKPOP3earZl9jWj3Kgu22tPT/MY5C3sl0ebRD6OvNLKDuGaC+u8sX04K1wllmX5u9rAm8yTUgbEds0oWG13Sv2oQedi9r5226Dv1fD1r1s3CvOeeaX0Bty7wW2+x55uuzsu/MBEkJjDJZ1mzsITY3xWJKLb4XwYsC7Vlp2+McEjrFwun+AXEcMKB7iMeo32zcKNH7P/JxirjvqwZ5/r9/HIDEb/vMZyRadJr7wK6xnU3nBoCU/UjZb5Tfahy2in65ngnYGSZQ5tTZbcbXpjraz2eRdPO+ze8+8OdpSy3HN/OI5a3MZDYGkTU3AsC73/NOAMA++6yec5x1wohGeWxoyHjc1kZtTju52HaeY+ePbJeMDce+/GUAgPM/RSa6M844IywbHOQo/Qk618atEpOqv58iVRdZJhFRkbhrPObpPl5js1gmQyY+z8g79xZB9zhmyMHBwcHBwWFJY1EC6qgxQTuArSrTfIwzYE9N08y/XBN3xQavBHv7SBw1Pi7RLO3qOdMuwqmZaZpJl9gF99Zbfh2WHfU8iurbv4xWFL6v6h2RaK9PCBYicPbnZr2e103XtFiltL54858tRXaPPMdN8Co/aMH0NImkzSxmYZ7FrGaGPN7Rbov4UbWf4X3mHocst6MpiURaqdHqrKOHRPPVmqzW1ux/6F4hoI4YL0hHEvjNb34TbrOiw8kZcjvVq/c8h5544AFyOz7yyCPDsjLn90knhOGxme+rHNk1lRCBuXVrjXMkb32dRo0ZQP5bP6WQJGFqTTtit3LKtqtDK2S0kZgBYL+166gueaqLHk/GeL/BQcpJNlmUd2sF05xqCDHtfezbsBayzf5uhP8rxtE07+OrMivut8xTXDFetg6lWDPP1IxWzg6M+caBluz04lzrkzHLqEu0ZnvF/kFaPdfUyn1wn8G9RkAND6hWZVweGqLI4QXOv6VdoyW3HjFJNto60DweWNgccsv6KEry2896e1jWxg4H55zzfj6nGodDAa8NLaEag2WEQoZodyEVZpfrwdOOeZFZf7c4o1edu7Flm2gO+TK3XLISAMKcWKGxFUYDQC7H4wY7Nu3cKZnp99mH8kwWc/Qt1qxRsUDPz7LAgDBQOd5fhxExXL8KZ6+/9NKvhGU2598555wDAOjuFkapVGJGjccnLaD2mGUrlYSlC9hs0dlBonkd/X31qv0RwEWgdnBwcHBwcHDYLRalGTIGSMU8PLz5wXCbte+ls2QL7m7rDcvyvFqM8gw0kZYVmPV0m86rvCOsB8py1upsp8xm1+5LK9CJCVop6MB0JtA2zN1hflYn1KEsaNX3yAxO0LQynK0jUMfZmfd8rvgt0cr9stnlvblCtK1q5tFAadu2XSDNOnNTYYsQBtbt3k7EfS3ssbu3en7MfNgccgCQ4KBqdnVTUXnlBvp7sGt87qrxiYaPAOVGFf3LJSP78BgxoNa2XlPPtXeQWK4JXp394IbrwzLLBBz7Esn8PDbDuhF+VF0ZeWZejLpvPGHzlkn/qsSbmSHN+llWpWHmMjDW3V7JVBBLp/gcdFx7v7jI/+lBCjbZ1UUMbywhffGWP/4XAOCoo44CAHS2q1xLVjfGf+vWb6saVXXwg+a61pQvvq2X3eIpdsrGJIzwWXUsynCvUGOwGzYgPNhumy98RquF6CwKC1ADztyxxLb7pjxMrOnYto2yps/kxT08nUwjXyliT8PzgHS7wdCQ5MPr7KDvgrUEaNdoG6phZITYApvLDQCiMXpvNg8XIHnANrC26JPnfSoss27f69cTC2H1MQDgBdYywddWrLV1yQ/sJzGYq4tpHqMZ4ftT7KLH3zTrfm9aBeBl5r2uXPLDXR4dM6SZYcuIWdZHP1PL9ljW7SmHHBqWbdj4MACgu0OYmvD8HFqjqNpYlPU8UQ4MWlLBkG0YCPt+P84BIAHgnnsp7MLFX6Icg5d8+ZKwzAaMTHOQV81ER5kFiqq8lknWodlvjs5o39WVwfTMfEGZBY4ZcnBwcHBwcFjScJMhBwcHBwcHhyWNRZnJGgEwVfXxxjefEm7bvHViniMIlhjMZlSuoSpH4FRWG/vbb2F9unCS6OAPfviTAICoYiyDR+n9bvFYzgjDaLiP4TkfSzyygav5OcxjcJsXs125W8nCW5VZWXBctcgiM8/2Weo61zDbvXrPIYCPfdatDv9uT5OIsFAmmryu3IKtOcm6jTZZEZm+HhoR88L3r7oGADDOucyaorja0Ab8t6feVtWKRPnvhioLa7NQme2c/ihvLpuhey1xUigbURYALrrkywCAFxzzAgBAPS9C4NmNIa7qZ+WS2mnc7l7lSqtYwOp+vOadgTAQgT1XC0MHyuEwuFDRbKsW3Gp/i/kcG2a/BBWB2kZ4V/tYcXG5zs4pyvxRLBebIlLvKfg+kM8H2G+/g8JttRY6YYskR5e2oVi0T88HPnA2AODCC9eH29IpepvFogonzrDWNxus/tL1XwrLykVuKQsYnJqsUWGZ/lxGmguVKcy3RS308nPQ6nXN5z8zT5kO1MyWRFgLa73FYNnOkQ/yKkWb9ZqvtwjKbpUpyhIOGy2n1Te7jUPqdHTSmLdzx645+2QyZArTQu1LLvka/bBhN9RjT9gurhUW9tHztmRSDiiX6/M+Tg3HDDk4ODg4ODgsaSzKtd4YMwpgy253dHB4YrBPEAR9e7ICrk847GVwfcLBoRkL6hOLmgw5ODg4ODg4OPy9wZnJHBwcHBwcHJY03GTIwcHBwcHBYUnjSTUZMsbkd7/XkxfGmC5jzA3GmL8aY/5ojDl0VnnEGPMXY8xP1LYrjDGbjDF38r+n8/aDjDH/bYypGGM+MM81v2mMOeTxuyuHxxOuTyyqTxhjzCXGmIf5fM94hGvebIyZG3XO4UkB1ycW1SfeyOe52xjze2PM0x7hmn/334lFudY7PO74NwB3BkHwKmPMQQDWA3iRKn8PgPsAZGcd98EgCH44a9sEgHcDOGG+CwZB8Nb/W5UdHB5XPJZ94mUA9ud//wDgMv6/CUEQvPwxqruDw+OBx7JPbAJwdBAEk8aYlwH4Olr3ib/778STihmyMMa8wBjzG2PMjcaYjcaYz/IM9488w92X99vXGPM/vO2CVisGY8waY8x9xphvGGPuNcbcYoxJcdnbjDH/a4y5yxhzvTEmzduvMMZcxufeyPW5nM9zhTr3S5md+bMx5jpjTIvY6004BMCvACAIgvsBrDHG9PO5VgJ4BYBvLuQZBUEwEgTB/4JC8sz3LH9tjHkW/84bYy7m5/BLY0wfb382rx7uNMZcZIy5ZyF1cHji4PrEgnA8gO8EhP8B0GmMGZy9kzFmszGml5/D/caY7/F9/FDd78u57A5DbNNPZp/HYc/C9YndIwiC3wdBYDOb/g+AlY/wLP/uvxNPyskQ42kA3g7gYACnADggCIIjQI3gXbzPlwF8OQiCwwBsn+dc+wNYHwTBUwBMAXgNb/9REATPDoLgaaCZ9lvUMV0AngPgHAA3AbgYwFMAHGaMeboxphfARwG8OAiCZwD4E4D3AYAx5nxjzCtb1OMuAK/mfY4AsA+kcX4JwL+idZiuT3EjvNgYk2hRvlC0AfgTP4ffALDJZL4N4MwgCJ6OvTeepIPrExqt+sQKANvUPtt523w4EMBXgyA4GMAMgLOMMUkAXwPwsiAInglgj7qyO8wL1ycEu/tOvAXAz+a5f4u/y+/Ek3ky9L9BEAwFQVABsAHALbz9bgBr+PdzAFzHv6+e51ybgiC4k3/foY4/1BjzW2PM3QDeCGrEFj8OKC7B3QB2BUFwdxAEPoB7+fgjQTP43xlj7gRwGqjRIgiCjwdBcFOLenwWtFq9E9RR/wKgYYw5DsBIEAR3tDjmXAAHAXg2gG4AH5rnPncHH8D3+fdVAI4ypJ3IBEHw37x9vufosGfh+gThsewT24Ig+B3/vgrAUXzujUEQbOLt1/wfzu/w+ML1CcK8fcIY80LQZGghfeXv8jvxZNYM6YDhvvrbx+LvS5+rASDFv68AcEIQBHcZY04H8IIWx+hr6+s3ANwaBMEbFlqJIAhmALwJILEnyJ67EcDrAbzSGPNyAEkAWWPMVUEQnBwEwZCtjzHm2wAeUSz9KOCCUD254PrE/H1iB4BV6vQredu8VdjN3w57N1yf2M13whjzVBBT9rIgCMYXWg9dpUdxzF6HJzMztBD8D4TKPPFRHJ8BMGSMiYFm/Iu99vOMMfsBgDGmzRhzwHwHGGM6jTE288tbAfxXEAQzQRCcGwTByiAI1oDu41dBEJzMxwzy/wYklv6/2Gk9AK/l3ycBuD0IgikAOWOMFdU9mufosPdgKfeJmwCcaghHAphWH4lHwmpjzHP490kAbgfwAIB1xpg1vP31C7h3h70XS7ZPGGNWA/gRgFOCIHhwgXX+u/xO/L1Pht4L4H3GmL8C2A/A9CKP/xiAPwD4HYD7F3NgEASjAE4HcA1f/79BNOV8tuCDAdxjjHkA5PnyngVc6ntMz94NoBfABXyNAWPMdpD9+aPGmO3GmCyX3WyMWa6ry/8XABzBwrdjAJzP298C4BtMy7Zh8c/RYe/Bku0TAG4GraAfBvANAGfZA7htN1WX/38AwNnGmPtA+o/LgiAo8bE/N8bcASAH1yeezFjKfeLjAHoAfNWQ8PlP9oCl9p34u07HYUjVXwqCIDDGnAjgDUEQHL+n67U3gTvIK4Mg2GSMyQdBMMeTwRjTHgRBnn9/GMBgEAQL6YAOexlcn5gfxpgIgBEAAyBx9U+CIDi0xX7tQRDkeaW9HsBDQRBc/MTW1uGxgOsTu8dS+E48mTVDC8EzAVzKA9YUgDfv4frsVTDG3ArgbiUEfSS8whhzLqi9bAGtZByenHB9Yn7cC+CbQRDU6BE9It5mjDkNQBwkYP3aE1E5h8cFrk/Mg6Xynfi7ZoYcHBwcHBwcHHaHv3fNkIODg4ODg4PDvHCTIQcHBwcHB4clDTcZcnBwcHBwcFjScJMhBwcHBwcHhyUNNxlycHBwcHBwWNJwkyEHBwcHBweHJY3/DzsqlFH7/CXaAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "import matplotlib.pyplot as plt\n",
+ "plt.figure(figsize=(10, 10))\n",
+ "sample_idxs = np.random.choice(50000, size=25, replace=False)\n",
+ "\n",
+ "for img_id, img_name in enumerate(os.listdir(INFER_DATA_PATH)):\n",
+ " plt.subplot(1, 3, img_id + 1)\n",
+ " plt.xticks([])\n",
+ " plt.yticks([])\n",
+ " im = Image.open(os.path.join(INFER_DATA_PATH, img_name))\n",
+ " plt.imshow(im, cmap=plt.cm.binary)\n",
+ " plt.xlabel(\"Img name: \" + img_name)\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": false
+ },
+ "source": [
+ "## 七、开始预测\n",
+ "> 飞桨2.2 CTC Decoder 相关API正在迁移中,本节暂时使用简易版解码器。"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Predict begin...\n",
+ "step 1/1 [==============================] - 7ms/step\n",
+ "Predict samples: 3\n",
+ "文件名:9451.jpg,推理结果为:[3, 4, 6, 3]\n",
+ "文件名:9450.jpg,推理结果为:[8, 2, 0, 5]\n",
+ "文件名:9452.jpg,推理结果为:[0, 3, 0, 0]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# 编写简易版解码器\n",
+ "def ctc_decode(text, blank=10):\n",
+ " \"\"\"\n",
+ " 简易CTC解码器\n",
+ " :param text: 待解码数据\n",
+ " :param blank: 分隔符索引值\n",
+ " :return: 解码后数据\n",
+ " \"\"\"\n",
+ " result = []\n",
+ " cache_idx = -1\n",
+ " for char in text:\n",
+ " if char != blank and char != cache_idx:\n",
+ " result.append(char)\n",
+ " cache_idx = char\n",
+ " return result\n",
+ "\n",
+ "\n",
+ "# 实例化推理模型\n",
+ "model = paddle.Model(Net(is_infer=True), inputs=input_define)\n",
+ "# 加载训练好的参数模型\n",
+ "model.load(CHECKPOINT_PATH)\n",
+ "# 设置运行环境\n",
+ "model.prepare()\n",
+ "\n",
+ "# 加载预测Reader\n",
+ "infer_reader = InferReader(INFER_DATA_PATH)\n",
+ "img_names = infer_reader.get_names()\n",
+ "results = model.predict(infer_reader, batch_size=BATCH_SIZE)\n",
+ "index = 0\n",
+ "for text_batch in results[0]:\n",
+ " for prob in text_batch:\n",
+ " out = ctc_decode(prob, blank=10)\n",
+ " print(f\"文件名:{img_names[index]},推理结果为:{out}\")\n",
+ " index += 1"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "py35-paddle1.2.0"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.4"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}
diff --git a/docs/practices/cv/image_ocr/image_ocr.ipynb b/docs/practices/cv/image_ocr/image_ocr.ipynb
deleted file mode 100644
index 95f6699855b..00000000000
--- a/docs/practices/cv/image_ocr/image_ocr.ipynb
+++ /dev/null
@@ -1,739 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "# 通过OCR实现验证码识别\n",
- "\n",
- "**作者:** [GT_老张](https://github.com/GT-ZhangAcer) \n",
- "\n",
- "**时间:** 2021.11\n",
- "\n",
- "**摘要:** 本篇将介绍如何通过飞桨实现简单的CRNN+CTC自定义数据集OCR识别模型,数据集采用[CaptchaDataset](https://github.com/GT-ZhangAcer/CaptchaDataset)中OCR部分的9453张图像,其中前8453张图像在本案例中作为训练集,后1000张则作为测试集。 \n",
- "在更复杂的场景中推荐使用[PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)产出工业级模型,模型轻量且精度大幅提升。 \n",
- "同样也可以在[PaddleHub](https://www.paddlepaddle.org.cn/hubdetail?name=chinese_ocr_db_crnn_mobile&en_category=TextRecognition)中快速使用PaddleOCR。"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "## 一、环境配置\n",
- "\n",
- "本教程基于Paddle 2.2.0 编写,如果你的环境不是本版本,请先参考官网[安装](https://www.paddlepaddle.org.cn/install/quick) PaddlePaddle 2.2 。"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "2.2.0\n"
- ]
- }
- ],
- "source": [
- "import paddle\n",
- "print(paddle.__version__)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "## 二、自定义数据集读取器\n",
- "\n",
- "常见的开发任务中,我们并不一定会拿到标准的数据格式,好在我们可以通过自定义Reader的形式来随心所欲读取自己想要数据。 \n",
- "\n",
- "设计合理的Reader往往可以带来更好的性能,我们可以将读取标签文件列表、制作图像文件列表等必要操作在`__init__`特殊方法中实现。这样就可以在实例化`Reader`时装入内存,避免使用时频繁读取导致增加额外开销。同样我们可以在`__getitem__`特殊方法中实现如图像增强、归一化等个性操作,完成数据读取后即可释放该部分内存。 \n",
- "需要我们注意的是,如果不能保证自己数据十分纯净,可以通过`try`和`expect`来捕获异常并指出该数据的位置。当然也可以制定一个策略,使其在发生数据读取异常后依旧可以正常进行训练。 "
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "### 2.1 数据展示\n",
- "\n",
- "

\n",
- "
\n",
- "\n",
- "点此[快速获取本节数据集](https://aistudio.baidu.com/aistudio/datasetdetail/57285),待数据集下载完毕后可使用`!unzip OCR_Dataset.zip -d data/`命令或熟悉的解压软件进行解压,待数据准备工作完成后修改本文“训练准备”中的`DATA_PATH = 解压后数据集路径`。"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# 下载数据集 \n",
- "!wget -O OCR_Dataset.zip https://bj.bcebos.com/v1/ai-studio-online/c91f50ef72de43b090298a38281e9c59a2d741eadd334f1cba7c710c5496e342?responseContentDisposition=attachment%3B%20filename%3DOCR_Dataset.zip&authorization=bce-auth-v1%2F0ef6765c1e494918bc0d4c3ca3e5c6d1%2F2020-10-27T09%3A50%3A21Z%2F-1%2F%2Fddc4aebed803af6c57dac46abba42d207961b78e7bc81744e8388395979b66fa"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# 解压数据集\n",
- "!unzip OCR_Dataset.zip -d data/"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "import os\n",
- "\n",
- "import PIL.Image as Image\n",
- "import numpy as np\n",
- "from paddle.io import Dataset\n",
- "\n",
- "# 图片信息配置 - 通道数、高度、宽度\n",
- "IMAGE_SHAPE_C = 3\n",
- "IMAGE_SHAPE_H = 30\n",
- "IMAGE_SHAPE_W = 70\n",
- "# 数据集图片中标签长度最大值设置 - 因图片中均为4个字符,故该处填写为4即可\n",
- "LABEL_MAX_LEN = 4\n",
- "\n",
- "\n",
- "class Reader(Dataset):\n",
- " def __init__(self, data_path: str, is_val: bool = False):\n",
- " \"\"\"\n",
- " 数据读取Reader\n",
- " :param data_path: Dataset路径\n",
- " :param is_val: 是否为验证集\n",
- " \"\"\"\n",
- " super().__init__()\n",
- " self.data_path = data_path\n",
- " # 读取Label字典\n",
- " with open(os.path.join(self.data_path, \"label_dict.txt\"), \"r\", encoding=\"utf-8\") as f:\n",
- " self.info = eval(f.read())\n",
- " # 获取文件名列表\n",
- " self.img_paths = [img_name for img_name in self.info]\n",
- " # 将数据集后1024张图片设置为验证集,当is_val为真时img_path切换为后1024张\n",
- " self.img_paths = self.img_paths[-1024:] if is_val else self.img_paths[:-1024]\n",
- "\n",
- " def __getitem__(self, index):\n",
- " # 获取第index个文件的文件名以及其所在路径\n",
- " file_name = self.img_paths[index]\n",
- " file_path = os.path.join(self.data_path, file_name)\n",
- " # 捕获异常 - 在发生异常时终止训练\n",
- " try:\n",
- " # 使用Pillow来读取图像数据\n",
- " img = Image.open(file_path)\n",
- " # 转为Numpy的array格式并整体除以255进行归一化\n",
- " img = np.array(img, dtype=\"float32\").reshape((IMAGE_SHAPE_C, IMAGE_SHAPE_H, IMAGE_SHAPE_W)) / 255\n",
- " except Exception as e:\n",
- " raise Exception(file_name + \"\\t文件打开失败,请检查路径是否准确以及图像文件完整性,报错信息如下:\\n\" + str(e))\n",
- " # 读取该图像文件对应的Label字符串,并进行处理\n",
- " label = self.info[file_name]\n",
- " label = list(label)\n",
- " # 将label转化为Numpy的array格式\n",
- " label = np.array(label, dtype=\"int32\")\n",
- "\n",
- " return img, label\n",
- "\n",
- " def __len__(self):\n",
- " # 返回每个Epoch中图片数量\n",
- " return len(self.img_paths)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "## 三、模型配置"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "### 3.1 定义模型结构以及模型输入\n",
- "\n",
- "模型方面使用的简单的CRNN-CTC结构,输入形为CHW的图像在经过CNN->Flatten->Linear->RNN->Linear后输出图像中每个位置所对应的字符概率。考虑到CTC解码器在面对图像中元素数量不一、相邻元素重复时会存在无法正确对齐等情况,故额外添加一个类别代表“分隔符”进行改善。\n",
- "\n",
- "CTC相关论文:[Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neu](http://people.idsia.ch/~santiago/papers/icml2006.pdf) \n",
- "\n",
- "\n",
- "

\n",
- "
\n",
- "\n",
- "网络部分,因本篇采用数据集较为简单且图像尺寸较小并不适合较深层次网络。若在对尺寸较大的图像进行模型构建,可以考虑使用更深层次网络/注意力机制来完成。当然也可以通过目标检测形式先检出文本位置,然后进行OCR部分模型构建。\n",
- "\n",
- "\n",
- "

\n",
- "
\n",
- "\n",
- "PaddleOCR效果图\n",
- ""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "import paddle\n",
- "\n",
- "# 分类数量设置 - 因数据集中共包含0~9共10种数字+分隔符,所以是11分类任务\n",
- "CLASSIFY_NUM = 11\n",
- "\n",
- "# 定义输入层,shape中第0维使用-1则可以在预测时自由调节batch size\n",
- "input_define = paddle.static.InputSpec(shape=[-1, IMAGE_SHAPE_C, IMAGE_SHAPE_H, IMAGE_SHAPE_W],\n",
- " dtype=\"float32\",\n",
- " name=\"img\")\n",
- "\n",
- "# 定义网络结构\n",
- "class Net(paddle.nn.Layer):\n",
- " def __init__(self, is_infer: bool = False):\n",
- " super().__init__()\n",
- " self.is_infer = is_infer\n",
- "\n",
- " # 定义一层3x3卷积+BatchNorm\n",
- " self.conv1 = paddle.nn.Conv2D(in_channels=IMAGE_SHAPE_C,\n",
- " out_channels=32,\n",
- " kernel_size=3)\n",
- " self.bn1 = paddle.nn.BatchNorm2D(32)\n",
- " # 定义一层步长为2的3x3卷积进行下采样+BatchNorm\n",
- " self.conv2 = paddle.nn.Conv2D(in_channels=32,\n",
- " out_channels=64,\n",
- " kernel_size=3,\n",
- " stride=2)\n",
- " self.bn2 = paddle.nn.BatchNorm2D(64)\n",
- " # 定义一层1x1卷积压缩通道数,输出通道数设置为比LABEL_MAX_LEN稍大的定值可获取更优效果,当然也可设置为LABEL_MAX_LEN\n",
- " self.conv3 = paddle.nn.Conv2D(in_channels=64,\n",
- " out_channels=LABEL_MAX_LEN + 4,\n",
- " kernel_size=1)\n",
- " # 定义全连接层,压缩并提取特征(可选)\n",
- " self.linear = paddle.nn.Linear(in_features=429,\n",
- " out_features=128)\n",
- " # 定义RNN层来更好提取序列特征,此处为双向LSTM输出为2 x hidden_size,可尝试换成GRU等RNN结构\n",
- " self.lstm = paddle.nn.LSTM(input_size=128,\n",
- " hidden_size=64,\n",
- " direction=\"bidirectional\")\n",
- " # 定义输出层,输出大小为分类数\n",
- " self.linear2 = paddle.nn.Linear(in_features=64 * 2,\n",
- " out_features=CLASSIFY_NUM)\n",
- "\n",
- " def forward(self, ipt):\n",
- " # 卷积 + ReLU + BN\n",
- " x = self.conv1(ipt)\n",
- " x = paddle.nn.functional.relu(x)\n",
- " x = self.bn1(x)\n",
- " # 卷积 + ReLU + BN\n",
- " x = self.conv2(x)\n",
- " x = paddle.nn.functional.relu(x)\n",
- " x = self.bn2(x)\n",
- " # 卷积 + ReLU\n",
- " x = self.conv3(x)\n",
- " x = paddle.nn.functional.relu(x)\n",
- " # 将3维特征转换为2维特征 - 此处可以使用reshape代替\n",
- " x = paddle.tensor.flatten(x, 2)\n",
- " # 全连接 + ReLU\n",
- " x = self.linear(x)\n",
- " x = paddle.nn.functional.relu(x)\n",
- " # 双向LSTM - [0]代表取双向结果,[1][0]代表forward结果,[1][1]代表backward结果,详细说明可在官方文档中搜索'LSTM'\n",
- " x = self.lstm(x)[0]\n",
- " # 输出层 - Shape = (Batch Size, Max label len, Signal) \n",
- " x = self.linear2(x)\n",
- "\n",
- " # 在计算损失时ctc-loss会自动进行softmax,所以在预测模式中需额外做softmax获取标签概率\n",
- " if self.is_infer:\n",
- " # 输出层 - Shape = (Batch Size, Max label len, Prob) \n",
- " x = paddle.nn.functional.softmax(x)\n",
- " # 转换为标签\n",
- " x = paddle.argmax(x, axis=-1)\n",
- " return x"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "## 四、训练准备"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "### 4.1 定义label输入以及超参数\n",
- "监督训练需要定义label,预测则不需要该步骤。"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# 数据集路径设置\n",
- "DATA_PATH = \"./data/OCR_Dataset\"\n",
- "# 训练轮数\n",
- "EPOCH = 10\n",
- "# 每批次数据大小\n",
- "BATCH_SIZE = 16\n",
- "\n",
- "label_define = paddle.static.InputSpec(shape=[-1, LABEL_MAX_LEN],\n",
- " dtype=\"int32\",\n",
- " name=\"label\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "### 4.2 定义CTC Loss\n",
- "\n",
- "了解CTC解码器效果后,我们需要在训练中让模型尽可能接近这种类型输出形式,那么我们需要定义一个CTC Loss来计算模型损失。不必担心,在飞桨框架中内置了多种Loss,无需手动复现即可完成损失计算。\n",
- " \n",
- "使用文档:[CTCLoss](https://www.paddlepaddle.org.cn/documentation/docs/zh/2.0-beta/api/paddle/nn/functional/loss/ctc_loss_cn.html#ctc-loss)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "class CTCLoss(paddle.nn.Layer):\n",
- " def __init__(self):\n",
- " \"\"\"\n",
- " 定义CTCLoss\n",
- " \"\"\"\n",
- " super().__init__()\n",
- "\n",
- " def forward(self, ipt, label):\n",
- " input_lengths = paddle.full(shape=[BATCH_SIZE],fill_value=LABEL_MAX_LEN + 4,dtype= \"int64\")\n",
- " label_lengths = paddle.full(shape=[BATCH_SIZE],fill_value=LABEL_MAX_LEN,dtype= \"int64\")\n",
- " # 按文档要求进行转换dim顺序\n",
- " ipt = paddle.tensor.transpose(ipt, [1, 0, 2])\n",
- " # 计算loss\n",
- " loss = paddle.nn.functional.ctc_loss(ipt, label, input_lengths, label_lengths, blank=10)\n",
- " return loss"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "### 4.3 实例化模型并配置优化策略"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# 实例化模型\n",
- "model = paddle.Model(Net(), inputs=input_define, labels=label_define)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# 定义优化器\n",
- "optimizer = paddle.optimizer.Adam(learning_rate=0.0001, parameters=model.parameters())\n",
- "\n",
- "# 为模型配置运行环境并设置该优化策略\n",
- "model.prepare(optimizer=optimizer,\n",
- " loss=CTCLoss())"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "## 五、开始训练\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 14,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "The loss value printed in the log is the current step, and the metric is the average value of previous steps.\n",
- "Epoch 1/10\n",
- "step 529/529 [==============================] - loss: 0.0891 - 9ms/step \n",
- "save checkpoint at /home/aistudio/output/0\n",
- "Eval begin...\n",
- "step 63/63 [==============================] - loss: 0.0830 - 6ms/step \n",
- "Eval samples: 1000\n",
- "Epoch 2/10\n",
- "step 529/529 [==============================] - loss: 0.0199 - 10ms/step \n",
- "save checkpoint at /home/aistudio/output/1\n",
- "Eval begin...\n",
- "step 63/63 [==============================] - loss: 0.0353 - 6ms/step \n",
- "Eval samples: 1000\n",
- "Epoch 3/10\n",
- "step 529/529 [==============================] - loss: 0.2133 - 10ms/step \n",
- "save checkpoint at /home/aistudio/output/2\n",
- "Eval begin...\n",
- "step 63/63 [==============================] - loss: 0.0259 - 6ms/step \n",
- "Eval samples: 1000\n",
- "Epoch 4/10\n",
- "step 529/529 [==============================] - loss: 0.0133 - 9ms/step \n",
- "save checkpoint at /home/aistudio/output/3\n",
- "Eval begin...\n",
- "step 63/63 [==============================] - loss: 0.0210 - 6ms/step \n",
- "Eval samples: 1000\n",
- "Epoch 5/10\n",
- "step 529/529 [==============================] - loss: 0.0110 - 10ms/step \n",
- "save checkpoint at /home/aistudio/output/4\n",
- "Eval begin...\n",
- "step 63/63 [==============================] - loss: 0.0130 - 5ms/step \n",
- "Eval samples: 1000\n",
- "Epoch 6/10\n",
- "step 529/529 [==============================] - loss: 0.0150 - 9ms/step \n",
- "save checkpoint at /home/aistudio/output/5\n",
- "Eval begin...\n",
- "step 63/63 [==============================] - loss: 0.0111 - 6ms/step \n",
- "Eval samples: 1000\n",
- "Epoch 7/10\n",
- "step 529/529 [==============================] - loss: 0.0039 - 9ms/step \n",
- "save checkpoint at /home/aistudio/output/6\n",
- "Eval begin...\n",
- "step 63/63 [==============================] - loss: 0.0093 - 6ms/step \n",
- "Eval samples: 1000\n",
- "Epoch 8/10\n",
- "step 529/529 [==============================] - loss: 0.0100 - 9ms/step \n",
- "save checkpoint at /home/aistudio/output/7\n",
- "Eval begin...\n",
- "step 63/63 [==============================] - loss: 0.0059 - 5ms/step \n",
- "Eval samples: 1000\n",
- "Epoch 9/10\n",
- "step 529/529 [==============================] - loss: 0.0096 - 9ms/step \n",
- "save checkpoint at /home/aistudio/output/8\n",
- "Eval begin...\n",
- "step 63/63 [==============================] - loss: 0.0061 - 5ms/step \n",
- "Eval samples: 1000\n",
- "Epoch 10/10\n",
- "step 529/529 [==============================] - loss: 0.0066 - 10ms/step \n",
- "save checkpoint at /home/aistudio/output/9\n",
- "Eval begin...\n",
- "step 63/63 [==============================] - loss: 0.0054 - 6ms/step \n",
- "Eval samples: 1000\n",
- "save checkpoint at /home/aistudio/output/final\n"
- ]
- }
- ],
- "source": [
- "# 执行训练\n",
- "model.fit(train_data=Reader(DATA_PATH),\n",
- " eval_data=Reader(DATA_PATH, is_val=True),\n",
- " batch_size=BATCH_SIZE,\n",
- " epochs=EPOCH,\n",
- " save_dir=\"output/\",\n",
- " save_freq=1,\n",
- " verbose=1,\n",
- " drop_last=True)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "## 六、预测前准备"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "### 6.1 像定义训练Reader一样定义预测Reader"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 15,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# 与训练近似,但不包含Label\n",
- "class InferReader(Dataset):\n",
- " def __init__(self, dir_path=None, img_path=None):\n",
- " \"\"\"\n",
- " 数据读取Reader(预测)\n",
- " :param dir_path: 预测对应文件夹(二选一)\n",
- " :param img_path: 预测单张图片(二选一)\n",
- " \"\"\"\n",
- " super().__init__()\n",
- " if dir_path:\n",
- " # 获取文件夹中所有图片路径\n",
- " self.img_names = [i for i in os.listdir(dir_path) if os.path.splitext(i)[1] == \".jpg\"]\n",
- " self.img_paths = [os.path.join(dir_path, i) for i in self.img_names]\n",
- " elif img_path:\n",
- " self.img_names = [os.path.split(img_path)[1]]\n",
- " self.img_paths = [img_path]\n",
- " else:\n",
- " raise Exception(\"请指定需要预测的文件夹或对应图片路径\")\n",
- "\n",
- " def get_names(self):\n",
- " \"\"\"\n",
- " 获取预测文件名顺序 \n",
- " \"\"\"\n",
- " return self.img_names\n",
- "\n",
- " def __getitem__(self, index):\n",
- " # 获取图像路径\n",
- " file_path = self.img_paths[index]\n",
- " # 使用Pillow来读取图像数据并转成Numpy格式\n",
- " img = Image.open(file_path)\n",
- " img = np.array(img, dtype=\"float32\").reshape((IMAGE_SHAPE_C, IMAGE_SHAPE_H, IMAGE_SHAPE_W)) / 255\n",
- " return img\n",
- "\n",
- " def __len__(self):\n",
- " return len(self.img_paths)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "### 6.2 参数设置"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 16,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# 待预测目录 - 可在测试数据集中挑出\\b3张图像放在该目录中进行推理\n",
- "INFER_DATA_PATH = \"./sample_img\"\n",
- "# 训练后存档点路径 - final 代表最终训练所得模型\n",
- "CHECKPOINT_PATH = \"./output/final.pdparams\"\n",
- "# 每批次处理数量\n",
- "BATCH_SIZE = 32"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "### 6.3 展示待预测数据"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 18,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "data": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAABmCAYAAADIx5U3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJztfXeYZVWV/Tr35Vf1XuWuqk50NxkBxYCojCiijMoIphGRZAIFE4ZRxgQMBkBFkZYxISgCisAAihL0pyM6oyMKAgJN51Q5vhzv74+9z937VT2qqxDoxjrr++qrqntuOPfec849Z+291za+78PBwcHBwcHBYbHC290VcHBwcHBwcHDYnXCTIQcHBwcHB4dFDTcZcnBwcHBwcFjUcJMhBwcHBwcHh0UNNxlycHBwcHBwWNRwkyEHBwcHBweHRQ03GXJwcHBwcHBY1HCTIQcHBwcHB4dFDTcZcnBwcHBwcFjUCC9kZ2OMDwBhLxRsq9VrVMb/69lVfdbx8neNha/7+/uCbYODgwAAK4rtqf3tthBvqy9QOHuhOttm17vMedzMe99TYN9Ps+cx32c089k0e1b+jN8NZU0OCPmz9/e4srV6szKDet2H7zc729MH2ydCIWn5Na6wxw2+rlTebR+wm0yTNu7goKGaVtAX5irbE/qEgUHIqO+ET9+JsAnz/1V9BP/2+T+pvt9028wyT5XZB0T7L1+2PCjbNrB15gkCeFxXjx+dr0bw5uNb/fFONauezc9gZu01F8yM382us9DxW2ow+0j7vP1dbHuiQ5adQ1Tr1Ya66LOaps/IzCgT1Jt8dT3PQ71eR30efWJBkyE6IIyu1rbg/6npMQCAbfZJdcZi1VaSEI9Jo50q0tYzz3hHsO2iiy6i40rUcRJROVetRL/TvK1YUmXzqPdCJyfNKLOZT3OufcoLvN7TBftIdfOyz08/o5mNXN97aMa2Zs/Bnquit/GBVW/29Vr5n4p6makWOmAyU2uoJwAk41Fki3vGUw4BaG9NBv9PTmUBALFwHABQqhSCsmiEfpf5wYQjcp6q/T48CaPNQr6GDcPEHAfW5xq/57Ntt36iHx+hOcr0s5k5hix4yjHHuzRNymy9WltkWzbbeKpUq/S+bKaO6h4woTYwiCKOVKQ12JYtU8U74+0AgMnCZFDmBR9ZqnwE0ikqPILobTV+ExUeESJGPhQlv8znonN+5P0fD8o+9Lmzeafg5AFao1TXaJmuU/OLQVmoyYSsDCqvozrzVEE7qQcjo25h9gNJZdU5v0xSFpnxW84g1y6p/f0ZfVXXwJ7DHl9XDdOeIcxH6NqF+Miaeg7VYFJoZu0/E7q7tLemAQCTPH9onIjQe43yxLnq63E+xPcgT8LwiqBQy8+6n9aWJKZz+TlqJVjQZCgSjqCvqxs3/+SGYNvK5f0AgFKOGrenZvz9/b0AgLGp6YbfABBtTQEAUh2dwbZSmT4a738/NVq/Kh+RyZEBAEBvOzXaWlVmQ75H2/x5Wf0WZhk0fpPZ5hyv3PCLqJpmI2WzazfZ5j9J1kszu56RGp27YTLEl9Pb6jPq76lCy+IYpjLCetVmd2Rar6qmMCXutmU7mKlnmyxRI+9VTOHA0AjXi87VuaQ/KOvsXrpHsG8hz6A1GcYvf3V3sG2fvfcFABR5slYo5IKypct6AAATU8SClsvSxiMR28HV0OXzNp8HfP0FDr6g9Rm/AWOPw8x9ABjuo6bGp9RP0v49ezUme+j2GZ5Rr7naeLVJ2e5HqOFT1ohmE55m7W72fvPr12bWgbJPa4J+T0xMBNvicZp0x+I05u3YsTMoe9lRx2B0Itukdk8vQiaEVCyFW//rlmDb8uXE0JTy1N4nJ2UytGqvvaisRGN6tSQfv0g8BgCYnpZvRyhMbc6L0O9YIi77R6mfFIs8mVHU2SCoz538lpMBAOmwTNbGNg8DADrjHQCApBcLyqplOpfvqf7F3bEa4oWaKpNFg/28Sn82dbuNdiqFZXyc2RT0mBtl2i+qVoS2+1f59CU1bFRD9YZ9wqrR2nPYsoYJv7E15slQXQ6s8Y41NW5XqvR32TJ/IZmkRPm9VKvUv+pV6f+jo/S8ly6h8TAWkalIsUCTl1RLYlYdctPUfiIxmQBXmF2KxOgc04VMUPayo4/CfOF8hhwcHBwcHBwWNdxkyMHBwcHBwWFRY0Fmskq1goGhAXR2ic+QFyKKKp4gemxqTCjd0TE+js0pqXQiKDNhouEmxoaCbX193XwdoiUzE2NB2fJlZCKpFokCCxuZxxVKu6bf52dCEzSz4Vs0M5PN3L/W7HJzmr+eaJnGrg1HzOqirp6frXrTOtt91P1Z5/cYU9CVsjIzsMnFC1ubs/C6NaYzo3FqKx1paUfVDFHjGzY8FmxLsl3Z82j/iUlpD9GoQbGy+x0kanUf2VwFq9csC7Z5/JDzBeoL4Yg82IlJoodLZaKCOzrlGWQDhxB1XzON/7ot2E2mmWlrpiedLuP+4lt7/3yfI11bm3bqZj4ee43H72nw5/Bcbyzh5xW8kibvIoB+Lt6MbbrvzbRVyHGFAvWJqjIvhNlEVC5TWVubtJ+Rkel5+U8+1aj5NWSKGaxcuTLYFouR2clw5MuyZdJf7PPP56lP6LHd3ntLi3KcYhN+hYN3RkZGgqLWVjJ9pdI0dmzevDko27t/FQBg26NbAAAH7XNgUNaVpm9PYZxM2q3tcr3WFjpXJi/ml1KZTHo1ayZTpmbf1t+ncdGo9uWxHcpw26lHyuo4NEBbr61pSvtUBsE6NqhIW9dtoMaM4wGgwn/O5esZjtG3Wre9GjukecqkFY+TKSzGbhFGuVfYb4DHtYjGxfS4YtmhAICd27cDAH76i5/Lcfy8DNsJX3XMK4My+35Dqg6ZPI2bUTaRtkWlT2SyuXn3iT1zdHJwcHBwcHBweJqwsNB6ULhzskUOGx0lZmfNcnJ8jYU7gjI70y8VyemppU2cpaey5EDnh8QR6q0nnQgAuOaa7wMAXnvsMUFZkR1NIzzzzGTFUTDVIk63gHJgC2q9APCMfe5Z4lPBDDXD3ztXnV3PGIdyNXOaax4dM8e9VmnOHVLsgF25Wue5clVWPuUsR2DUaP+6crZLJHmFkZPVg111gMMwky3i8Fguz5/PeCrheUAiIWwQAIyO7wAAJDnCrK0tJWWjtIqt1WnVWKsJq+Y1bTMz1jXzjL83M52CGxpoI8PR8NqDNqq3eo1lZlaJWsZqR+3GOMW6r1b3exC8OR5pY5+YMTY0KWvOzXozrtPkRTeTm+B239PTM+tc2Uka/zxvrli43QMPHuKheAMbsWHDBgBAGwfOdHTId8I6U/s8noRVGHGRHa67e5fM2t8GHHS0tQdl45PExlpH6lWrVgVlLd3Edtz3f3+h86jvxsolxGIV6/TNmp6S70uUHXMRlu9eJEIvzPOIOQk19C8u4/4SqqnIJ4/OEapzWXh247PtRBFkYH/ohm22z9kzhDVROZNl0jICfuO2ZsN+ld/F2Mh4sM2+s5VKrmD9po0AgDvvvovqEJF7Peoocl62Y113d3dQ9uB99A5WrlgBAHjT618flBUL9M6t0/xvf/2boGznIFkH0u0ypkbi9K4rNXoX2hrR092FUeWsPxccM+Tg4ODg4OCwqGHmspfPhGeMH/GADY/9Jdh29x23AwCy4+wLUZQZdZpXAV6EZm41IzPrPM88W9u6gm1jk1P8F814+5YIk3TkC18AACiz3Xb5MpnV56dnrIIXzMAsVDBk1+efy+dod8KGWDYNGW4Sit8M1qYb4ZVSrSYz8YjXeOKhIfEJe+SxdQ3bqnU57ujXvAYAsHLlXsG2DRs3AwCS7FuUy4ucwgtedCSKZaBW3/0CcyEP2LnzgWCbzyuTECuEVhU7ZvtbOs16K2rVEoloFRGLGW2toW1bSi88a19jctDw53y3ekk5Oxw42OY3YSEs+xOE62v/MXvfdM/1enqOOuw+zBVaP38xofn7/M13aPDr9Px0/7JyDX39tKKuK/XZ9rZlqGH3iy56xvOjiGP9w+uDbaUCscIxFtayPk8AEA4xA9ZJ34LR0dGgzN679kUZn6I+0790KQAgoYSY7LH2CWSVBSHCLIxhVqa3U74hNda9i4aIjdafxZFxOmfvMmHoRjLE8Na92eJg1qcuxGH04Zr0mzBv89jBpxATDZxAlaSJT8985ByafXP8OVh/M4MhAkQ2JZqmZ5pMin6aZWoGhgeCba1p+sZ3LyHmbuu2zUHZ737/ewAimdDTLd/6o48+GgAwOUbMk20DAJBK0rVzORrD2tuF+QtF6P1oRr1UobY0nmF5n7Dc7K9++/9wyVe+jK3btu2yTzhmyMHBwcHBwWFRw02GHBwcHBwcHBY1FmQmM8b4YQBDAw8H2zJMISY4fDgZE1NYOMJUIOfOmMyK2m7PUnJY27JTwiJbUkSjW8XVbhV2fMtNPwYAvPP0U+lcY0KlRprR9wtCM7XYx1eQbhpaO3PPJsrVzRShm2MhNrYnkxFvdt3Hr3NFqYBbJGNEYyb4t1YPteqzPm+zoZAAcP3tbG5VtHb/Ugq/3baTaNmTT5PULfsdcCAyeR/V2u41CYRCxm9tAQYGNgXb6izlX+UcMlNTU0FZkingnm6SihgZFikKcYZV78HMIRsxS51a9YMQm9+atrkgJp//lT5r6vacMbX7jPM3nJNNOB63BaPahP2bzWV+vRN7IsTZfBdrQ25qwV5PWCpDUJ+j9VonXd2HpqbITaBnCZmIRkZkHFyz+pA9ykz26IOPBttC7PnrcZTAuKp3lM3D3ZyNYMuWLUHZXqxOrRWo4y1iugGAR9etC/5uYbPNgQdS2PzouJLjYEfeMqd7CiuV6VKew+A5R1ksLtfIsQxGxRPTXijJ0iHe7GRG1jnamseiNelf0Srdq8f7FKPyTbQmqlCTLlvh5lTV1usZx2mV6Znq0jWdFzRIUOnNOs6eK8cphPT8IBajumvF73KNnkmeVfY9Ze5qTdEztIEwDz74YFD2wP1/BSBO1S99yZFBWYklJWzZY48puRV2L9D1Ctm5R4rG1um8tJXDX3Q4sqUiavWaM5M5ODg4ODg4OMyFBYXWe4YSTdrVCQCEeSbtMxNSq8mMbZpF8rwwrSzTKVkZDg+R45RRq9koJ/br6qSZZFHldIpEaUa4YwetpKNhYRUioTkcIBlNp4XByk5PxRvzuvvNjgyOk7mk5KOxq8e/hwV6olm3dj23lZxjzXIqN7uuDcOenXvHho7bkEYAyLGMwtQktxGVZjvCq8MYh5dWy1KH1x1HDtR5lXzVrkCs0NiVV34nKPv0pz+NSy/7dpP6Pr2o14FcFti5U3JE2UVlWxu1554ecRwssaPm0CC1/5pa6glT1iQBUcAQNYmftfmOfOWA7c1YGjZjiHwrAKdyy9n+WFfnssyQ3abPxQ7TnpUAUKtniSVnB+qZMgF7CPzAg1TqJ06l8rxn3A4a+u4slmieEghzsMWTHD5vheaAmWH2jexqPG5QKO3+yA3PhNAabQ2kVQCgVqH2m0oRc6PFIm3rswJ/vb29QVl2msaRSkXG+HYOyNm6gwT7Nm3cKGUc/m1FHvXz6eomy8NYcZLrJG21o5u+Lxm2XlTDUta1hPrv9nHp47UwS2OwA7Ue7/2alWCgfqmNBPVAnYK/m/rzMkeou92voppZzbMMFP2v0qMhwueqB/tKmZxj9pUs0ZVSMiYWVWbW9HfZfvcTnIHaKJ0KG+wU4nylvV3iCH3AG48HICxQqSiJcet8zp3bSaJk79VrgrJMnvarqOCbCrNTVtZF95d8MT/vr6ljhhwcHBwcHBwWNdxkyMHBwcHBwWFRY0FmMh+kv1BVOaHarAbMJDtSK1XheIzoKmsmK5WE2kolWYFU5RWrVoi2K5WJQmxNCV36T/90LADg1/+P1Chf/rKXSsXmMJPNJAIb1WYtgTZb0aEe5GFSeYRMo3msrs4uuc8sRdqkTs3o9aCimsxroub7uGimRtHMIZy2lZoqntJxBs3qQKgpDtaaFYayrPYaEyo6HKd3F46x86Ayg0Ys18vtoJQXarTGeWysNg8gFPckJ7l7zxlnBmXdffti1xnpnib4wJIeybUUjrLjdIa0t8JKZR0+qeAWC/QMO9uljZfLTbR6wA6WgUekfrf2vOzQqJ2ewX83U4a2fxvbxvUwMOOcqs5e4LCtdrd1DfSlZjtX29xMdbNb/XofH4FNTOk02S3KjmF4q5mpyK2PnMOpupmOlz/HM+nqIvPM2Jg4AVs3BLY6IRKJzjpud6Pu15ApZQNTFQAYdqItsLpwXNXbaszYQINkXHJYWtOH1hkq5Mj81sL7vfa1rw3KrG7Xf//2twAac2vVy3T+173uBNqgnHAnpuj7FWdtvIm8BD1MjpO5px6X/a0ztc/ma/0WazbQxrZ/5ToC1oXyWZ9Lj2HWedl2Ie1cX+G+mlcO1FU+wD7lSFUOCM8wk1XUuYozzGTajGcfc6VoTU+qPXM/8ZXulTWLRdhkp81kFT5xLkvPsqdTzGT2PcVj5FNgc70BQEcHtftShMaW4eFhqUOI7jYSl3lGiOcc4+P0PSpV5FzRcBz1JoE+zeCYIQcHBwcHB4dFjQXnJqNVkRw2OUGz5t5Oduyryqo2X6QZYZi9uaJRUQrNsSPp2Wd9JNgWilF5mT1mYzFZnRYydJ2rrvwuAOC2W24Oyv75FXsDANqZpdKriDyrWNqQzj6V42bjenK80w571jl8SR+t9McnZIVQ5ay9ZV5tnHa6hHrnSzSTvvWW22iDWgxY578k598aHBQFzxUr6TpaUTOb5dBAXkkmErJSCvEqysoP+Gr58J3vkINxnD14PZXs6sQTT6JtPcTI2TxAABCK0vvMZ8UxviXB6w1eibz7zDOCslQnSyDkaP9MURwlOzlP0Hf/k+oypXLbJA2tBuPsQJ1QIbI2oD6mVJhtXiK7ArSryj0RIU/aap4dByNheoY60tn6mifjtPqpV9UqmIMRPOW8+Wpe9SZaZq9bCjna9vOf/o7PJSvxuqH2UWBnRxvmCgCVKjFytu14Rp55ncOBjXKgfvfbzwIAjA7TubQqbSJOdZjMkDPrLbf9ICgrlWj/GGerrqk2Z9u0VafV4eM2FFf3Y7uStNnL3/CGN8j9VBpZ2LDKIXXzzTRO2LbTrC+FeP/xcWmrLQlitTXzYv2//TIHi6gU4i8/mvIotnO29I3KqXe/Aw4AAFx3/bV0vYi8y7YOCh/euplyd+mxKJPnd6ccQu1jCjOTqp9Rseg/4dCLpwSqbvb9NlNZt+8hyEWoYN+bzmt1xBFH8Onp/Ev65JkNjhCLYNvorbfeGpT5dWpDv/glZUi3/RMAnv2851NdONw+1CIUjB+jwTxXlXHuqh9eCQDYPrANANCrgiTiPM6d9qZTAACdXZLLqzRO57Ci9GVI+4qGmAXj/yuqv/jMhORUgEKaFZ3Ht5Cj8Ze/cGlQhinq46kOGo93VkWy5MyPfxgAsKSvnyshH6vcBO2XDjUGRgFAlr8PCZU7LplkR3XucpNTwmKed95nAIgz+8iIyOjEYtQPk630zS4VpQ8nEsTOffzfzuVryLyhxh+kUlHYnljSjrP0vCIxaWPlahn1eQYzOGbIwcHBwcHBYVHjCYkuDmwTUaywTzO63AStqsoFYRc624mFSLXS79EpWdmf+s73AgC6elcG2wbGeEUdpVnjhGJl9t9vHwDAEAvwfePyy4Oyv95LWe4PP/xwAMD0pAjZ7b03sUbjbHcPKeuutSeHdZg+rxYmp2lVFgrLaqV3GeUDyuVpdn7Cm/41KBsZpfu/665f0j37IvxUKlsBK5q5dqscLaNjgwBkBQ8AnV00W27l0HXtM7B27VoAsoqKK3GwUfated/ZHwCgfFAArOTszQ8M0fP733vuCcqmp+h59XaJ9EGG/YF+cecv6FwqM32J7eTL9lkNANiyc3tQFqzAeXX0za/Ie0pyPp6lKVrlDW7ZFpTF2qiNhNS7sOGTNWYwQoop3Gv1oagAqO92gbmQH0EcA0OShylfomecaqO1RjQqbERumpZQ9TLdb7mgV8O0/0knHx9s6eyh/a/+wRWzrn3aKWcDADgtIK69RlbBufpDAID9+B3t2Lk5KOtZQu1rcJDaXjQs9etoI6byVS97Y7DNr9HKLBaiOre2SsboSrBapr5rwtL/L1v7HwCANXvTOTMl6f+WzbGZsPWq0QrsWcE9ve1f/uVfADTmtvrTn/8MANi6me7xlFNOwUzccsstABoZKJv7yLCDhWYf6swCD2wVFrcjTf3DdoVjj3lVUJaM0zO67dafAQBSfRICfyDfRyvne7rj7juDMptHKfCVC8n6tMRja0gJ2dnq19kXT2et712yZo8QXQyZkJ9AK/74xz8G22wew5YEjVflgvgLZpn1t6xyPitj4bve8U4AwKPrRMDxuYc9FwBw1fevBtAowHfCGyj7uWUjbBsHgDt+/RP+i55xPCnt+NrrfgQASPfQOx5RDMdzXvw8AMDNd/5XsO2xrST0uO9B+9J1BnYEZeUMtfP+dmJePnj62UFZVwuN/QVmbhJpsVRkd1BHTrEXkBeXsTDXQdv+NCTf3gTLAXzvksuo7lMyRn/yPfQNCDGrft63vhaUDdWJVfn0eZ+l8xTkuFZ2LkoF+RSlvyxbTQzX+of+GmzzQpwpntvqV75ycVCWzxHLFGHx5aQaN0aGyUcrlaL+n5lWofVsebr4oi8BAFpUmH+JWdlieTYzVGbfIF/lJjvw0ANQQX1e3wnHDDk4ODg4ODgsarjJkIODg4ODg8OixoLNZCEAWx5TJgGmEyMcRrhSObN57Kz4+//33wCA//z29+U4Dp//0tf/M9jW0kHHnnLKu+j4sDhCjQ4RrdbDJqZvfkPMBjf++OMAgLe+9S0AgLByHPbZY9Xmw6qVxVHLOtlZZy4AyGWJritVbDiwCg2PE1135MvIWXL1mv2CssFheg4/vZ1o8nhe6PVIlHO0sSkslRbT1iRLEmgH1zamah+6714AwM033xiU1Tis0Tp69/dJSPdJJ53M90V0vFU5BcQ5tL6EaNmEclassgN0IiKOp6ecfCIAIM354Y4/8fVB2WODRNX+8KYbAABrnnVAUFZgirhWoOd81de/FZRFOZw8zHmAlqTFXDjJYfYh5fxase/MOo1G5RmtXHPwHmMmC6MVg8pMVgOZfGIJus9KRTkhTtH99XUdAgCoK9/fk95K1PZ0VtrOj278BgCgVJVcThaxMPWXt7yRaPh0izhqXnn9p/na9D7qNTFfDY+SefLQZx9K/w8MBWWnn0rnSsX6g22Xful7AIAlnURHv/a17w3KchlySm3rpLbW1yft6tLLzgMAFIpstk6qfsnjjnWg7uwUE63tl0NDUi/rMG0dqG+8UfqENc1ax2ntiHvkkUc2nP8nP/lJUGadqYs1av92jAGAVAtR+l1pMZ2Bx4S3vIHGmb89KDka/3LvfXQuDvvuVCY3JGgce9HzD6P/VfjxL3/9KwASQFFSqshI0PN6ppnJPOP5EcTx8F8fVtuoStOT5PrQpkwmbex0Pj5E/ebUU08Nyqo8Xuvv1A030LhjzZw6V5Y1jx111FEAgK4eeQ85n96vNZlOZ8SpuLuH+tIkB4XE1Rh9+ZVr+XhxoC6H6D0V2EysHdnBppzuFJlKp3dOBkXvftu7AQB7ryT3jVxBKdBn6F5bWC66qmRGBpJ0zupecj/nXfwFAMDgAzT2XHuBmMKWhaifZLh/ba7LvZ73bTKr5bns/Pd/NChb00Z1TvE3Yd26R4Ky5cvoGfX2qUwSgzSWWBWFT37yE0FZlZWhp6fp/ltb00FZMkF/7xwg0+BBBx8WlJ16CgUmrdyLntGGDRKM0NNN37tiWUzuYibjwTQkbeWAZzszmYODg4ODg4PDvLCg0HoLy3QAQC87ClY47G7HTnGKbWGnwhcf8SIAwAuOOCooK9dpNrdphziqeTGa2X3xC5cAAD7wIQm7P+AgWkkPDZBDXDgqVX8rO0z+6u67Gq4HAEnrhMYzd98Tp2LrOD00KM6bbZ3WSZJm4skWWcG8/JhX0321kuPa0Jg4aifT5Ag2maHZ9pIGgTV2IOMV7+iorEDb220Ir8xL1z9M2X3vuINCQKtVHXZIq9lPfvKTVIchqXvvEnoXO3fSM+rqklVED+fXGfNolTo1LnVvjdK7KBRk9XDjj2n1NZkjx9XhnLynFxz9EgDAm95Oz/3UM94pd8oCWJ0ssFXRIdMcOhqNWnkAcZA3vLTQebI8FrBTEl/Y0+DDoA6vgYWzYc/VGq1cNTPk1xuFCx99WBztM9Ms5AZhRMPs0B+yrJgS9TP1KJ8z3HA8ANQqtK3GyZoTSVmV2aCCTZvICdQ6/wLAf36TVo0tEWGG0iyzwJHeuO4aYWXPOptY2fEx6vePrZNVXJqdQ8P20ah0R5btKHJOorwS4JycpGei81e1cD+0LFBR5bCrVun5WkJYly1fTgEaVtBPNcfgmj47UO+33/5B2cQYrWZHRyXcPhWnOvztb7RaPuiAg4OyGrMBnZ10z9s2iaOrDYToaKGxZWBM5bjiIIcUM95F5YhrnrFrVQODUPBuASDO/duOgZpJsQEiVnzRiioCEopfVQ7zcRZjtWyRlnqwTKPN4aavk0pRe8oye71m1b5B2RAL+3Xw2J4tixP3+CCN1+84+3Q5Vy/1p+0jFDxy++23y/6jNLYODBDD25vqC8q6WDqhzOKLlt0CgLYE1S/JTszTSiwwzw7nwyPClv75EWLenrNiFQAgkxH2t8zfvRoHoUSVjMkkd2TDshHdfdLXJwap7kn+Hh166KFB2XSG+oJmbFNs0TjzjLcDAK78nlh6CswuWfHcYkHux+YpneAcljqnXlc31WfHDuoLe62S3GT5LJ3DKIOWN0voVH8nPMw31+cztbc5ODg4ODg4ODwpWLDooucBeRU+X2K7YDuvHhMdsgIt8UxweJhWQtM5YWWSHWT76+yQ1d/IBK0Q9t2fyrTY2PrHaMVppde/pjKWn3oqzfAPY+Gs7h4JVxwcoGtbEcFkQs45MmbTSYj41iT74iSSVK9XHPPPQdk9v6NQ0aP/meowOCLMRh/pvA4VAAAgAElEQVSnF4nxceG6rPirPDsPcchfokVW4iFmScZVKOd1P74OAJDPW2ZBZtRJnolv3UEr8URCCRcys5Pmd1BUx3ns+9TdSSvQbYqV6e+n57V1o/i9ZMaovG85lUVaha0YZDHNaU6lsmzliqBs3WMkHve9b1PY68QWCW29/Tbyj3j/yeQTNjWt/GCiQQIE2cYrP9PErW1P4ogMQqgp559ahVfEIVoZpdvExh6P0LsZHqZ2dtll3wjKwiH2fQjJe7Nig11LlA4/Y2yYVmrRCB9XE9+Jf+Y2es/vSOph6/a/BWXLl1MfWL2a+tkWDkkHgBXLSMJidFBW57k81SczSe8moUTNxkfoHbakqE3fecuvgrJN64lB6epmMdWc3JdlA6wP0LJl4vu2mevziqNfGWyz2c5bOBw6nRJpf8sy2XB9Lc5Z5KzYMZY3eMPr3xSUXXcd9bN4mu7noYfEx2Up++JVlTzFiae+FQDQ0UZ9aHC7pAn41hUkBvuB970PALBivwODMvAqfcc2Wukm2sRH8XXHkYzCf916M9+LEsCcp1jcMx3eHLdp/T9LitKz0gnf/S4982JetVX77i3zkhfmpcjsSjxGfXB6XMrSMRq3x9j3c+23RBLkgk9RCHopJEzXBMuRPGvNQQCAxHHyTn/2M2KJJrmPRyJS5rHkyGYWSlzaI/IRYAbZZxHEeFTaQowZ4i9/9cvBtn0PIl/N8U3ManWID2aizIwa/x9XYfpFFnNsSdLn/8z3nhWUfedzdP5Skeo+nZHvRDhEx+Vzsq2zg1iv/mX0+9TTRNaij31frS/fe9/3/qBscJjYpf32JXa1Nibvwo4NfUupD27fJmxpR5u1dszmcYKMRXrbrL0eH44ZcnBwcHBwcFjUcJMhBwcHBwcHh0WNBWetr9cBnSjZ+iMXS0SreSpPkHWaa28nCjLRKgTWFDsvptrEpFXmrNivPY5CxOvKKbWPabiTOeSbhWsBAD+85nwAwCkn03E7dwqtlmAFzrqNz46pUEZ27EqnxVQ3OUV03VnvozDnH90gobjrN26i58C5nPr6lwZlBaY4SzUbki/3GuWs7jbEMBaXOWihRM+he4ko1o6xubC7m2hGzxNT2Ps/SDS8dZzuWyrOeVbV0zrNFvJilli+nMKut2wm81p/p0ggDG8nU+LeS8XcZZWxNz1MTrZZT87VtS/t95GPEO25cUied28fPZPAvKNMRO9+J5nHvnoh5dD5hHKQH1Z09ixYp+o5MoLvTvioQak5wIStSiq924rKaeT7LcFRAPDIIxK6mmDphphypk+nySyUL3C4va/L6B3a6Gqd36evl96DdRxOpcU0O8kZuqeyVK/WlNDxA4Nkjm5NiDJ8nh2aV+xNbfS4o8+UOrRRnescnl7Iizlj1YqVfP+c9duT+u3/rGcBAHaySWz71q1B2ep9yezdt0TGBisNYR1iy8o51wYk2JD6/n5xCB1jZWurLl1XWczt+TNlMi+vXrV3UOax6rlJyfMeY8fYiMeBFyNiJvvIJyig4VMfoTZ94cWixDu8ntST21hlXduFptmBtMzvrmcvkUcYnRZH62cSfPioodYgcWBD67M2b58Kre/qIvPO6DC9qwaHaDZ7aUdo68D7xjeSSrrOP9beTuZT61ytc6H99Od3AAB6uC088pD0vVZW8u9kJ/cLPn5eUFbnEI78tASYdLIz9uQOanu9KQlWGdlG43ciSn1jYLsEuZTZUfhZBz0HADA9KmH3YTbqVPPWzCsf2jiHjw+NijtFiFWfO9lMPKWc/dM1+u75LOswOSZ1TyRoLEgkW3kfcU5Pchb5WpH6vJaICHH/Taq8bQUOcd85QI7kq1eL2W9klMYsqxZ/9dXfC8psFocz3/shAEBPj5jJW3jMy7BzebvKjADpvk869syvi4ODg4ODg4PD04SFh9YbwKg8VYapIZ9Fw6o1WbGVy+w4zOHcZRU+F4vR7M8yEADwoY+SUFyCnZ3rKsR4+w5aOZ7GuWq+/c1vBmWvfOWxAIAfXns9AKBSljq8+YQTqF7MWI2qPF9LmcXYphy0VqykML5f/5qEIpevlJnuMOdOyzGrVVfh8140zvdP9+gr4Scb7mkFwfJ5cUCLxWmWPTwsz8GuYn2e+U9PC2vy2fPoGbUz42LFFwFhFs46i9ijjk5hvOxqqiNJ91xQx6UitBrYvlFW58v3XwUAiGeozsW43OvJ/A4KEbrH3m5hmeLsqPo+dsr7ybeuCcpyU7Q66esmhqGak/ckYcR6fu7P+L0nzt19UOimPB8rHVBiwc+CCjiolej5tLVRzjC7KgaAAq+W6nXpXxF2LC/mGzOzA0CSWU6bb0uvnrNZetZ2hRxWbFOYV8sTU7T6W7JS2IhQFzmXbnlMwme72khc9NijKRdfKi6MzcgQtduOLmpD//pGySZ/9TUkVtfSSvWcVE771qF/CYvd6Vxjj9z/ANVTORNX2JHZsg2eEkNds5oYHet4OahEJHu6mf2x+a+UuOMYr6SrYTp3LSzPPcR59DwjQ2SU65PL0mp4//3FSTo3Qte+8IvECH3pws8HZR/l7N0+5zvr7BVauzBEfcCODRM7pe5aiuCZBAODiIk0hNZbJ1o7tuncZNu3E6vQwbnJrrrqqqDsTSy2GVZirH1LqM1Y1vNVr5IccdYhP8Zt/GYlzpnNUBsbrxIb098t7T7LYpA2d9jAiLBy4xlidnpXCXufLdKYvCRN2zZs3xyUdbOYbL1GbejfeMwGgEqe2tj2rRzY48l9hQ2NDSEeSmrSHOHzP6tXrw623fPg/QCANWvICXlvVWY2EWNVYQHHeEKChGw+wEdZzPAlq6QdD7MsS1+ajtPh+rkstfF0ShirtWtJBsd+szPa4Tpsxxy6x/EJCZjhTyIuueQiAEA0KoFXn/r0hQCAbI7Z0l6xftQr6qE8ydgTvy4ODg4ODg4ODk8bFh5aHwJKZfGBCDEjZEPrfZ3dnENpjaFVqhcSv4Uar6S7u2W2fflaEnw78WSSLO/tFx8Wm4E9xv431aqyc7bQtY877jg6tyr78Y9/DAA4+W1vo/pitu1Z+xjY0P2/PkCr080q7Lh/ufgUAEB7p6zwRsZolW3F4aJ1mT1v3UrnOORQmsEPD0v9bMaR73xXhOymM3SueJxej07V8c53EitjQ4y/+EXxTbA+UN9k1uztbxcxxPZ2zg48QSuynnZhJLI8Y08n5P3UeRWfZxs/jNj/r7icVvxv/xj5VU2o0NYKM2OW8bDZxgGgJUxM1Yn/SukMrr/62qDs2JNOxONjz52zGxiE4aGk2Mh6mJgDm63Z+q0BQI5XapaxsQwObaT9Qyr1QI7TA6TaZosu5qYzfC7qczW1lIy1MrvKrFEsLMdFOWQ3ysKlU1Pia1DMsi9anwid5Sbp/d56G/WlV738zUFZH/edySnyn+nukhWeXaVXq3SPOit8lX13hlnsrkXJTaxcSb5GmzZtCrbZDPaW/dEpKjZsIDmHVatWAWgMrbfsgV0N6+dtmbQ4+4tY5kbXL6zano3utuHvBZVd3fZ7y3a+j0PsAeCS88mn0WYEGFTsTyRO52ptYTkMlWYgD+lXzyT48FHxKw0+Q7Yd2nejx2E7VlRKNC5aYVkAuPvuuwEAxx57bLDNtoteFjBMp6XN2fPneUzSPkNJj9pYkX1ylvdKe/RKVL/tm4mx6exR6ULa6LhsUVgSywR77J/Z36WEC1lyJRKmduWpz2y1Qu2qjdmjsBo3itNUry4WwYV6Dn4t23DvAPCc55Df0eAj5Ac6sEPYrNUhvjanWKopX96pCWLGlrHP67p164Iy24cqU2QtiapUJ6GwtfTI9+uMM84AAKRT1vqTVftTf/oI+9FpqZyJCer3bR20rarY8DL3vbYOYgqtZQUAIk9MJ3pe2HO/Mg4ODg4ODg4OTwPcZMjBwcHBwcFhUWNBnFMIQEfVw4qQOBWXakS5TY8R9RtPKf3HJFGA2TrnO1FUuM/5V0ZGNwTbli+l/GPfv4Kcqs56z2eDsu42ou3LWaLtzjntK0HZNT+WnGcAYFQm57edSJR+kcP0vLBQj8k2okJf/s8nBNt+djvlNxtns1f/UjGNDewkij7OSsHRitxriKnXFnb2LarcNqv3pue1fSflK2pQA+XQehvSDwA9Pdb0QOaLcz4sIehD7LCaSBCFeO6/fyoo+8IXKItxiHOA/ee31gZlZ59N2ciTERtyLbnJwgmqT64sVOV4YJ5hpd+w3GukQBT01RdQiPwHzvlAUFZj1fGWNJ3z3HMkw/mV36PQyvWT1FYGe1U4fYTVg0tiJjDsNFkL0Zy9ojJ614A9Qp/Xh0ENESSi4qxuPLqvbIHMjxFPnKt7WR19+0bqNyoxNepWcVs5Yxs2i5m6bdMqN1mN87nVE/xbKO1aiRyFbVur+NLPKhw225Emh8vilJiqvSI981JY3k2klRzr87U/AwB+/ttPBmWvfPGHAQD9nS+meyiIWeIVLz0JAHDPn24CAAxkROHZmq2iMRoTxsclxDjKwQgHHiiOnevXS84zQPKRAUBXV0/DtlpNtVU2k8T4OWRzKuAgSdeZLHJW7aSYRkJsqihOK5eAGNH3+Uk2K6dknCnVqK/aDOq1olD7H/oI9YHb76Js6X5VjpuYJpOKZ00jVemDXpjOkVN52zrZIXycFeJDqgG1phPIZmXf3YsapgryTls4uMMGt7Qm5VuQY2fkJCs1a8X5AptIUipPnc/O83Y0GJ+Q6/T0Lmk47tjjZWy/9Y7/AwDkszT+7BiW4xKcHSAUo9+RFulLm8ap/Uf6xBw3wc+9wkEMn/+6fI9al5D57f3vIlNpMS+mo6VsDs2xeXiqT+6rXGSzLZvewsq0XeExsKroizJ7WnOXxY/uFImBc950KgBgiNv7WETaRT5Jbcz3qV5t7dJnJ4v0jevy6F2MDol8hM03Wa3J9yvJ79FmjJ/Oe2p/utevXPYjOvekmOMvuoi+VVVW7s9kpOzii8nh/JJLyDl7TAU9IUquM8kOlXmBTaKhKF3b5kQDQDlJd52wHoBjhhwcHBwcHBwWORbsQB2BwVRGZmqFCs0u40kOGW6XGdtEhqb43SlydCuoUMtojPaPK6fFfJH+DoVpVn7t9dcFZWec8W8AAI+dP9NqZWGzVNvwVJ3teedOcipbs5qE3EbHxaH3tce9BgDwlz8/EGwbGqEZamsrzWonJoRBCcKUmXnSzsHWiS+foefRmZTZtnUS7ezijMhZOc6GJFqxMEDlbWpNNFwXEKdBWy8tUGYdqOv12dmcrTNjjvPl2BUUAAwMkzhWu3J+jfh0LsPh84Nj4vSZ5v0MO0Fe9JUvBWWf+gzN6ofHrACkrHStc6O9hze8QfJE/eKmXwAAjn75McE260iZ4TDcggrHTadDmMo+dWGW80XYC6E93h44bAJAqM6rXxaWq9WExdi4kRiOKChcPRaXdlIu2RB5ed8tLfwMMjYsVdp2qrWnYf9QSMqqFXp/pSJt61oq8gebNlGfKMVoZdwWF+fPMq+3Q1Gpg2HHyelshv+X9/C7P1MustNPoDYwOihjQ+8SYkQnhunZtPfPbuM2r5rOUJ9qJ1brL3/5S7BtFWeutk7Puk/YnGQ2IOLAgw4KymxIti3TzrbWuTqa5nB9pZw5zUKTMSPvJxKzIq10Du0gbMPIsyzM16WCMjITnEOO+2cF0m77+jhsmJ3tt22XbPfLD+rn60g/tg603/8+SVZEwlKH6enCHsGWAuREbfs5AEwzk2WZum4lcbCJQ7zrIXqPLSp/5PHHU+42vWr/2c9+DkAca193wvFBmT2/FeIMKQfqlz6fclfe8ytqszFf3kOKmcPRDI2rUwPijLz/PsRG/GG9fCfaV9O9DfB1EsoBOG3baI7afWdcAm0MO9h38/dyVDkHd/N4VxklpqysGOIQh7rvs5eEz/83949/2psyy1tRWwC4/htXAQBedzIFq3Sn5HvZwvksrZxFTrEytTK9g+ERGhuWLpOwdmtN8JXy4QQLBO+1imQKOiPyXq2kiGVltTP7i170IgDA/feTPEBeyazYR/kIC/4ecMABcr1JlvIpqkAtfobxkMgHPBE4ZsjBwcHBwcFhUcNNhhwcHBwcHBwWNRZkJqsDyKOGkidOri1sFguz6u5UVukZTNFcq8zaKtmczL0iTJ0lkkIhZpnSr7Lqa0XRcY9s/CsAYOUKMnflMgNBWUcbm1bY2XTnoJQd/MIjAQBb/0bOm35dnBeXsHLy8573/GBbqo30H0pFoijDEXG49kJU5zY2f5RVnpQy04tWQ+fWn/xHUNbTQ+YM6xBttZIAIMLPTTuJWdOIpTG1qW7ZMnKAtuYCrcFgtTwinDzO5gcDJLdTJzuiF1S+rBbW0ZjIiXpoip3fxtgkGm1TGkTs2FdhGjcWFcfTbJF1RNic2dstOWesTksmk5t1X1b/RSvN2m2d7HQcUc6h09M17H4jGT3zQiGLel0aQ5zpYKuFE4ko9dw+anMxQ3RyWenK+Lw2Uf7/gQlIdHW8WWWex7Q8pO/Va1SHOOdHmhgXdfGl/WS+8qv0jstZcUy3ppxyWUy5pcI0l5FJpjUppqbNj5JD/7atpEsSj0lbSLVSX/KMNfdKe7T308P5waYm5Xp/Y42vNWtE62iCda+sKbyidFOsM/YSPtdmpcViTcWtbCbQbS7QHmLV/GmVeyrJmlvhmjhfTnIdQ1XatnHD5qCsbRUHlXDbLmTkflLWtM1mBhORcxo+/wjndtr/wP2Dsp1TpB+jTYj77ksBHe94xzsAiPkfAEaGJ3DTzbdhT8HkuDxr22ojbBa15mIA6GadoRirMb/spS8Pynq4zJo5NaxuzR133BFsO/roowEAy1eQ2Ua7Mizn/GF5NvnrfJD1LLWvNqsN1immuofuJ3PUi1/y3GDbvRv/BgC46QZy5ehXudbe+LrX0znYOXhqcHtQlqrSPba1UV3iYZXos9xo5PSUFlOU+/jWjdK21ywnPa4sj+07tm0Lyl7zGnIBuZEVuJ934nFB2eB26qvWXHbw3vsEZVb/yea8NMrwOj1F/fegg8Rs1Zamc0xO0ndCO33b75zVj9JO8/Ydv+oY0o/64Ac/GJS19tKzXL+Ogj7+ep+YJ//lhLcDoHyQFnbMslplRgUV1Pwa/Hkajx0z5ODg4ODg4LCoseCs9VUALe0ymy1zKOjoKM3AyxWVtXoNOTIODtLMtad3v6Asw3lHJiaF2fDYGdBjZdzpgjAVBxxCjmMehymHWmW2NzlBKzrLpCxfKhm3H/4DhVN2tNMqoEc5koIdW9OtsvKqs0O3nW3q/EjjvDrNZfL6cDpHmpxD48zKlBVj09JL156eohlrNKJCctkp2KqBAkCeV+LWEbqzXdizwQFivaxzsc7pVOKw9gQ7p0fDUvelfcTQDOygGXxVOTav2Z9Wm8NbJHfM1DCrBi+jupd9WYmP8qrX43BU7Rg3zmzOyuUr+J5ltW1Dng07Lh6477OCsmUn0P3fdNNNwbZ3nUnZ0Qc5U7OnHOQ8D1Cp4XYbfNRR9ktoUSrh0Ti9h0yO1Jx1/rg0h3OPDZJzYCSuFI6ZQa2pZz2VodVY1xLLUMqqZ2yIVt41UBsKR3UyK6pDpUbMU12tpDgKFrUyt8ewMD01GzZbE6YrFiZH3lic2lqlImzpiuW0gkx30KqskBPGcYLrDg7rrag2l2Nn+tFxbo8qVP6gQ8khNJNXCs+cydo6Kmtl3Cl27LbO+q1tcj87mCWOx2N8Hbkvj1nIQoaeVX+fOIsWc1S/ScUkLemh8iIrT3uQfpzZSddJtbRynYSRiPE7T/IquqjGhkKBzvXlL1MY8WfPP0+O45W1ZXUBIMrO1Ok0XUcHY1x73TXI55/CtN4LgAHQ0ykq90MDxCC2sBq7DhipsAwBE++IJ+T7YtlhncOvzhkGivy+LVsIAHfeeScAcdDtWyqO7IPjxND0LSNW1jr/AkDY53GUMyhEVD9r47b9h3t/F2y79qc0Tg2zw/XH/v0TQVm0Tv14eRe1l+w2yVrfn6RrZ0eYJUzKfRke761jvq8DIri/LFXP1LRTOy9M0rNduUy+IdhJ36o3sLTAd3/xs6Cogxkhy85ODEn9fDZ3DI7Tt8Cyp4C0uaEhsbzY52W/l9Go1NnmIkuys7hR+QStirtVZbfyGAAwwnn+jjySJHP0+73hhhtomwo8aUnR2LD/gTS/SLbKWLwQOGbIwcHBwcHBYVFjwYk+agYo1GSl63POqrYusoEWizKjLhRp9vbes0kYMBSSFeVRLydbYUyt8N7+dsqKPcnMy3hG7MSDY2QrjSdpBvr1y78WlCV8mtl28qy5WBC2ZMUyYonqPAMd2CHZ4a+/nrLc22zhABBhEbh4gmabIyOyKmvvJJ+EN72RhBxNSB5fnVmP66//IZ3Heywo28a23KVshx0bl5l4iUUGTzxRcnNdxgJe1ndIZyO3jFCQBVrNqO1qa2SEzm/DkQFgcJDue9kKsg97yo/lkQ3kT3XueSKmZ0XHvnoFCTdOZmXF77WyLTjw25BVfUdPN9eZVgEXXHBhUFYp0P7Ll9MKZtPD8oyetYoEN3M5YQO2bKEw40KZ89gtEVYvFAIqe0AcsU/cUPAeAcAPsVAcr1jCKvy5u51Wqgn2nfj+D74dlJ34llMAAPW6POv2DmItq1XLkhpVRu29XKFre748u5t/fjkAIFfaDABIt8uqLMzh4hu2EqtwxqknB2U9neSD8t3vfi7YFo+wX8AonSui8tRVuO/U6uzzlZDn8MMfUzuOJFhmQTGINnu89UOaUBntp5iNue46kdY49dTT6e65L2gpgy99icL6A2ZI5UCyWb5LLG56660iTDfALGuIWSMtzmYZTb1i/faV3wUAvP44Wm23JGU8s0yVFZ/rU+H9g49R/4pz3x1R/f++B+4DAETYp+msd0l49EVrbUZvYUost5bhPFaZsozFhUI18MvbnTCgVbbOWm+lEOw7qhaEHQsxw2ZZOy2bUGPLgx4XrB+YzXbfkhJm346PS/uon4UVO15up/NuzXNerHadd4vOOTpM5yyoUPnkSupnXTVhKHJxKv/Iv50LAMgUpD0u66Px7dd//B8AwAHLVgVln/7a5wEAp73lrVSnvPJJZd/LCPvd5CryjGr89yc+cE6w7b2fIDHeQ/ppnN+xWfwC905Tux1n5vHFzz88KPv1QySeGmMrxgUflnOaaWq/9ltVUXWwEh7aX2d83H5rVvL/Yl2wshFWssKO54DkGrTvWsumVHlg7+rs4TKpg80dqutQ5Ubfw7nmiirfmwkEYHYNxww5ODg4ODg4LGq4yZCDg4ODg4PDosbCFKg9IJoAyjUxK1kKq81S00aoxBonUslliP7s7hE686EHyYF03XoxleSYavRCdP6//u3eoKzKJoAwK+O2d0rVvQkOyWfHxEpJaDIbsh7mEMbePgn1ttR8VDloWefmcomuF4/JdVqT7HCZJ8fQRFxCBXcOkkkvworNceV4XSmx7ACHuuvcO3HOSdTD+WwAYPlScj5+9FGi17/8JVF4/tjHPgZAQuq3bN4clOWzRJ3H2ARzytvE/GHp88FBUlat1MVJt6ubnOSsoycA1Dyq5JtOJJXoaFro3K9+/TK6/1a6//ed9f6gLMH3PTxEdOmyJf2qjO512waiSztS4py3Yf16AMBJJ70t2Pbn+8mEcNAhh9HxCalDpbJn5CYzAMLGRyotppnJDJkk8yVWYI1KvTMZ2jY+Tu2rUzn7pjif23RObB1VdhYtVSwNL6RvLEy0uvHoXaZaxJySrz4CAKgZDrtV5uHWJJkbDzr4n+h4X+qQnaQ28Objzw22JdJkTrv2+q/TdVKyhjrhtHdT/VhaoLNL6j5VoPYbDVFbGB+R61gn0ba2EP+WsaHKQRhr10puPasobFWNtbr6F7/4RQBiStFmlle8gkJ462z2OPdcuS/rmBlhp+eBHaI6/JVLyMQ3MSK0f5pV6UtsHo6FdSAJbTN87eG/PRSULV9F/Xmcw49T7RKGPc0O0JVpKvvK1yTH1eg0mXP0OBNik1krO4nW63veetaDQTKcwI5tElLey7IHEdaNGFJSIqvZZOL5VKZlE6w7wF7LJR/mjh0UGr5q1SoAjfIi02zOtyY3rYBf6qHzl1rpmU16cp0Yyx2YXnrHmbC044EctYELv3ZxsG28Qm1t7Q+uBACMqLxbnDIMa3rpvd/401uCshZ2rfD6qb2nSiqTQobG70KF8xF6Ku8c95d4tzhQP3d/CkDZyd/Sni4ZTwtsRu3qpG3TNfmOJdjUPDlIJq6IygLREqE+sXEDfZd7e8VM3JaWulqcfx59C6wjfzQq1/nCF8nUbgMAli+Tb++OrdTXrBxGtSR1+M63KYfl5Dh9n3XQQ7qTTG819f2aZid7a5bNq9xk/rwD6x0z5ODg4ODg4LDIYfQqalfwjPFjHvDoo3cF26JhmuFGIzQDnZqQWVy5TLPMiy+hFd7GDbJSSLAA08S0CLEtXcGzUMOsx45Hg7LDnksOifvtTyuEI458YVD2HJ4t2xnoMmZWAKDAoeeFPJ3T5hwDgEEOKVyzRjLTDw3TOawDcGuLrOI2baEVybvfTSHfdeWt2MLntaF/flWcxewqeCcLq3WpGXyEmahxFeaZ5yzHt91GKwq98vnEJyiEc906Wg3cdZe8i4EBFhPrJpbpTA5NB0Qwzt5PvFXYihyzD6ed+Y5gW4XD5qucrb4allVKMkUr6RFmf1atlFXbwGa6x6u/Sc6mXkFWZpUpWq2s6KUVQrWsQsg5XLlThY6Oc519u5oKyUr8sMMPx1S2imqtPl//uKcExhg/hDCmpyXcdCpL774Oqn93t6yutm8hZmh5/7MBABOjOuSd2tNJJ78+2NbZQ+uVq39whb1iUHbaycTIWX/ca38ggnum7Y8AgGW91NZ2DEvW+niEVmOlaXoPp/3rF4uMrvsAABmiSURBVIKyCFjUU+VtypcpACASo/vJKXHOJT0kblcqUpv9xrdEbHTVIdTu81liSZKtLwjKijna37bLsGJZSkVqF1Y8FAAuv/wbAIC7776b95cV6Pe+RytJ6zj9LuWEnOOs3T/5CfdLNd5ZZmiCV+SaeWxhNub8z54XbPvsv1PevTKHgr/mVccGZVas7lbus+19kvvviOeQVMAYM0OPbRbBwVyO7r/KbLsXFkf3eoTegR27AKDIrEk61c73I/v3dK+G7wO+P8803U8Rwl7IT0dacdcddwbbLKMX5nE1pt6tHbfrzAB0pCW/1TGveAWAGe+NmdY7f0H5DLVz9QknkHO7dbS3QnwAcPkvqQ0M7KAxqlexLOU8tZPuJbRtw1bpL9fcQHngyjHhDkoe1dVwbkmbtw8ACtOcRZ3lIlZ0SuDHmaeSaGA6Rm21wxfmPLedc1j6xK6HleN8sY3u+U/Dm4Nt8W5qA9/5MjP1k2Kx+fzHKWhpkNvcNXffHpT94o/3AAB+cPX3qQ5VaUP+MItPtlH/0vn3LDs7OiaZ7G+8kZ6ptThoccwVK+g7fN555wFoZHNtPtHTTz8dAHD11T8Iymx7j7LVRPfLsay1HgWbgqz1MZZk0A7Uzz3iuaiiPq8+4ZghBwcHBwcHh0WNBTFDIWP81hCw7tF7gm11nsUW8yxWGBHmpbePwlrvv58Yni9/+atB2QSvCLX9PJMllqhYoVn21d//VlCWLdKM1bDI02/v+XVQ1l6mlcFhh5FvydKly4OyHKeHsNmxsypEfDkzQuseejjY1sHhyjY1QrUqK+R4jGa2p5xyGt2DEtOzz/FHP/oRACARlRBjy/RY4bB8QYQI43GbOkNsziv3p/DmLewzdMUVVwRldvVrxRZ12P3ZZxNTYENxx8dEMM6yRWWeRYcT4ieV4bDwRIf4vZz0ztNpWycxf5N5EXcL84zdrs7P/9R5QdlqFrwc20o+Kn3tsiryOCt7le3ZMRX2WozSfTT4C3DYdZ2XAVrQc+8DDkMVu38VbIzxDYCRUQlrrdRpNZZIUhuyfkIAsPbrVwMA3n8WyRhEQrIKDjPz5YWFCXj1a19K57ILXF/WLwVOb/Pzn5EYXL0q79REiDmcYv8lHUbs1+k6mQlqz0s7RQz12GPO4LrLuzGG21GI2m2hIG379tuImZycoHtsSct4UsUO/k0ryUxOVoZ2lWjFFpMqU7llZfN5uY71m/n85yk0+Xe/EwE8m0Xeth27ggWAW269GQCwF4fYr18nbLOVomhppb5hJSkAIMptuyUhzML5n/ksAOA/zr8AgAimAsArXv7yhuO0T0ORU67cz2kdNBueL9LY1d7BqVGqKr0Ov2rNguWZJbbsiJZt6GhfsUcwQ54xfgxhXHGZjFtHHUUCelZYUIdSW6FKy4BH1f1aGYg3v/nNwbYc+4hEwtTea2rMsO3Kirfq8P7hML2H/l7yOxkdFhbD6hsmEnS9td/6RlA2OEntKdqmmA1D1wyzLIMWCG1P0bv0uG2XJqXs3HM+Sn9w+HioLO0r7dN9J9lnSI+FtTS952FP2J+WHho7RrcQK/3di+X7WmN2qp1Tjmyclrb92a+QZEMri/MWd0hZT5Tq7ofo+IlJsVhYOYyQSndhWb0LL6Q+ob9H9m8r7jjW5Ht00UUkNmrZYEAYoZFhurbt3wBQYB+wZIu0e+vDF2UhxoKSOTnsBYehUC2iVt+1BcExQw4ODg4ODg6LGm4y5ODg4ODg4LCosSAzWcQYvz0E3H+fUNTWZGJNSJmMUiq2ucY4C31VORynOB+Yphf/iynt449/He2vQvjDnN3dqmBqp+J0SM4xGzPne/K/33QuyNuYafZNk338ueaQnHkcZbVt5jNWMrFBgq367HIzj3fTwIg//r1aRDnnlA2dBwBWQIDyo0OZ40MrvE0l70aNn4mtXkjVIcz5taIcyhmrSh2inO07XLfHqzw2htqNVtstM83ss/nIV87DBx5yGIpVoFbfzSYBz/iRsMHmLWJ+icap3sUSmUOSSTEBFQv00L5yCSlPhzwp+8iHiUL3jVD7lSqZJ+MJehHaAmIVJEImzb+FcjdgqtjjvuGJaRaGgxzqrHBeV7l8fH7+RvoXPDKBeWwaQF3McagxhV235m7Vjj2qgzFUh6oy/6md+Lre7G1aO3ZGn7v4YglztmPBZz9LZqyG/GPcflta6B5tWDYAXHklhUUbj9WHlXlt+VJyJP/whz4cbMtMkqn+0ksvBQD0KUX0415DyrjdPWQKr9elDtasYPPXbdwkzrl7rSazcpmdPssVefelGj3vpMr27Rs7DlpTouy/etWBqFaB+u7uE2wme+DPkm3cBnq8gh2itemjylnNh4fJXGPzGgJAnk1iIU/MtiEef6zjdDgkA1ec3SGsaVGbybQJBwCMP3sc9nlQq3lSVjN2LJRtpTrVOcJyJGVl0vL4OnYsXKKCQtY9SNnu99mLzLajKqQ8wU0mXrX1k+oV2XKYU7m/ynzbVgUgIVVAhK9th9+8PD5UuD2GmhyX5L+rYX5uTRNAznfbTDTp4/5cfX329yvL/SSqJG9qPIexuTJ11vqDn3MwSvUK6r4zkzk4ODg4ODg4zImFM0MA7r3398G2Gq9e7Ey/qhwHbVk0RtPSmmKGrFNpXc3W29vIQevmWyh/0NtOOjUo27aNVnQ2X4kOa4+EZEXHtzXHXexi/sezUllbLZQZ4hqokNdZs+YGxqcJM7SQdOxNV9SP9z8Q5zB6rdVmV0GKxEHFskWeXSmpSwbPlzaqBRPCdZqxC0OkWSMu4/1DqhLZVnKMi8eFpcixoGeOV47VmlzogEMO36McqDdtfjDYlmqzwnjEvGxWwpjRCDEo1om5o00ypdsQ1Fhc2s4HP3QWAKCF88GVyyorOQcvtHLm68y0rII9y9hYRigkDvDBNmaIGpqjZY28otpWQAPqwmahRn3Wq7AjuB9XO9oT03urxXZgNp4YM6RX+Tb30ac+ReHEF14o+fAsuxxiD1md08wKPY6zg+vSpZL1e8dWqqsW7etst/nU6F1MjIpz6c03E6v9jndQ6LRmeCybbfM8dSppjRw7UNvcdjpDN7jvhRTzkWUmyJJfNiciAOy990F7DDMUhYdNj20KtllHeZsbTr+/F76A8mYdeOCBAIC//U0CWro76JkXlTN9Ksl9iL9dVrgWAErMBNkyXwXAtFsGNMixpcBjbp1/aya8zuNjA5vOR1uLhQ7vt0Euk2MsGVCSOhx4OEnCrPvDH2jDMmGNYtXG3yFVwQq38ZL6rNjx2uO+EdafEP5t76MS8mYdZ/ePK2YoxlWtNGVxm3yXZn2rmn275vqGzocZkuPD7CSts9bb0PpCKc81kDoc8rxDXGi9g4ODg4ODg8N84CZDDg4ODg4ODosaC8pN5gOoAWhraVfbiNOz/m0hX2j8StVqAhCtnFMUdc8SMiFo7Zhkgk1UFaJER0ckn9KyfqKwPUPnGhsTXR6TnEHNzWnGqs1Rpsu9Gf8vEHVRzZztCK3r6zfZxzTZz2LGvS2QEa8yb+orM6M1VTakOWJq2PqPew2X8Rt+a0dosI4I2KSlcyfVbBlfuqYcGK0iaakknC0z60iyDolWoPbM/PzLnw54YaA1LXpZYywJnWDH6d4+cQg1oI4SjZBT7MaNolT+yc+QurjNowUAF/zHZwCIVtUXviBq0ckEmaSGRnc2nBsAErZD2v5Yb9bmrKemXM/47PjvK5rcn9EO69pUR/vV2Um6cd/GtunX5zD/Nn2X5nHLm5nJrAaRNTcCwAc++D4AwF57rZx1nHWuDYfpeGvWB4CWFuq/2qHf9r0MO/WmO2QcPPY1rwYAXPA5MtGdccYZQVl/P6svx+hcG7eKJlVvLylV59kkHFJK3BV2DNcuARU2i6VSZOLzjLxzbw9Z2hoYREwkyAsJAEnOY3jsq18FAKipfFi//OUvAQCtnKcv1SZ9KcJtXAfalH16T1YkPQql1MzPLMIO1LGEyreY5TGJ/29wVuBm4fNYVlPmH7tNdUtU2PwaZoftlqS0hVFW5m9ns2q4TWxbm9mBuquPvmdDdTH/WSucbaLa7GXbQEO4TDC2spsDNLyGe9VtKOj+1veh4Tr0uzqzzzdg9rnmdqC2ZV6TbfY8c33HZN/pcXKvMcpkWbHaQ+yWEIkob/EFYA/pPg4ODg4ODg4OuwcLcqAOG+O3AtiqMs1HONvv5BSthosVCeGt8ay3u4ccBsfGROnSrhRSreJMOD1Fs/8Chxvedeevg7IjX0IKpr1LaJVdr6t6h0TZ8mnBfByc67Mz/M4ZkmiarNybX7zx36aOp48/x43xisZvwvQ0OEmbGauoOSbumhnyeEe7LVQPq/0M7zP7OKS5HU2KOm+pQjP+ti5ymi9XZGm2at+D9xgHanhAuSxtcGCAFGFznH9Lh0ZLziRa/VoVXaDx3i1sbqAlPaTY+p6z3hOUtbAj6TnnfITPqdpc4MBrQ4bVstYyQsEScVehsjPLdUOx7zc04/8mZ/TKszc2DaNtlLeYXS4K7ICseq2jsXWMBoBMhsLhbRDHzp2SmX4vzpaez9C4o1mjfI6en13dA8JAZXh/HR5uuH4lzl5/+eVfD8psLqdzzjkHANDZKSxCocCMB9Oc2oHa49V5oSAsnc8UbXsbOd5qVd+VK/aFj93fJ0LG85OhGH7zm98E26xz+sQ0vQ+9es+yHMujj5I8xRFHHBGUFTkPXDImDI/NfF9mFetETIIurPxBlNXt9XVqFWbF+X/9lAKShN+DDsRuFpRtWUT7vqwSMwDss3oN1SVLddHf2FHer7+fcpJN5KXPW4dpTkmHiFapqFupF9lm/64FvxULbxr3qasyG/BimaeoGhpsHQqRRp6pEc0CgBhzfRubWmwWFlofj1grk6j62yv29hPLWlEMb/9e/c6B2sHBwcHBwcFhPliQz5AxQCLiYf3mdcE2a/NOpsk/orOlOyjL8sw4zKuyWFJmmzb6cyqrcvGwP1CaM/Sm22WFt3pvmm2Pj9NMWotwGV/b9XeFuVmdwA9lXjPcx2dwfL+ZfbTJcXY1OlcoflM0C0luDHlvrBBtK5s5fKD8JnblGWduKGwiYWDD7u1EvK4de+zuzZ4fr/JsDjkAiLHQoF3xl1Reub7eLgyNzWZSnm54HpBsNRgYkDxH7W3UByzrqUOjbQju8DCxBTZHDwCEI/RcbB4uQPKAbWDfovPP+1xQZsO+164lFsL6xwCA51sWlq+tGDobku/b7u/P9otpbI+M4F2qVaPH/deG35tmYqPMMlZVSH6wyxNjhvSK3zJilvXRz9SyPZZ1e9ZBBwdlGzauBwB0tglTE5yfQ6bzJWlzYfbnCbPgW0EJv9rwXvt+P8MCkADw4EMku3DpVyl31GVfuywos4KRSRbv0wxDmFmgsMrhF2c/NNu/dEb7jo4UpqbnEqB9elCHj2KtjN6lkpF9cJTao/XBqqixprufmN9xZvF+fPONQZllAo595bHBttFp9hvh5tWRknHEi9C7icZs3jJpq6VoIzOkmXDLqtTMbAbGhtsrNxVEkgk+Bx3X2ish8n9aR2KTHR3U/yMx6V93/vG/AQBHHnkkAKC9VeXks76U/L/ugbaqYVWHut9Y14qKxbf1sls8xU5ZTcIQn1VrUQZ7Bb5ou2CNg4PttrkkZZqRMzMoLECNM7O/r3YsbcjXx75/27ZtAwBMZ0VGJBlPIqv68FxwzJCDg4ODg4PDooabDDk4ODg4ODgsaizITFbzgclyHW97xynBts1bx+c4gmDJrnRK5VUps6qnstrYv+tNrE8XTxD19bFPnA8ACCs2zn+C0e8WT+aMMFD+fBLP+WTi8Q1cjc9hDoPbnJgZttrMLbxZmXWBjKoWmWeG2z5LXecKZoaS7h7U60A262OffQ4ItlWa+AlbxFld2spO6PiFj370bADAxRevDbYlE0RX5/NKJpZhrW9WmPvytV8Nyop5fmrzeBEN1qigTA8NocZCZQqr26ImfpCz0IxxnytWYI4yLdTMlkRY5rzapGG0crR2VqVos1Hz1SZiu9YKrywcsMogzcanFpYPaWsn08jOHUOz9kmlyBSmHbUvu+yb9IcNp1aPPWathdqabB89b4vH5YBisTrn43w64aOOvdasDP5vTZKzea5IL6uq4tStOcnKCzRY1vlZDQzL8/zRNdcBAMY4l1mD2reV++D/PTWClW0wAf9fU2VBbebrej7rQUvjTqfoXgucPNAqjwPAJZd9DQDwsqNfBgCoZsUReGa/jKr62eaug8bt7mWutNKMV/fjNe4MBEIE9lxNDOIoBlOD+QZXNBtomu1vMVewz8yXoBSobdYDtY8NQilWOWBLmcnzxXyDIvVccMyQg4ODg4ODw6LGgkLrjTEjALbsckcHh6cHe/m+37M7K+D6hMMeBtcnHBwaMa8+saDJkIODg4ODg4PDPxqcmczBwcHBwcFhUcNNhhwcHBwcHBwWNZ5RkyFjTHbXez1zYYzpMMbcbIz5qzHmj8aYg2eUh4wxfzHG/FRtu8oYs8kYcx//PIe3H2CM+R9jTMkY89E5rvkdY8xBT91dOTyVcH1iQX3ibXyeB4wxvzfGPPtxrun6xDMYrk8sqE8YY8xlxpj1fL7nPs41bzfGzFYn/QfCgkLrHZ5y/DuA+3zff70x5gAAawG8QpV/EMDDANIzjvuY7/s/mbFtHMAHAJww1wV933/X31dlB4enFE9mn9gE4Cjf9yeMMa8G8C0AL5x5QdcnHPZwPJl94tUA9uWfFwK4As37xGuepLrvsXhGMUMWxpiXGWN+Y4y5xRiz0RjzRV71/ZFXfXvzfnsbY/6Xt13YbMVgjFlljHnYGPNtY8xDxpg7jTEJLnu3Meb/jDH3G2NuNMYkeftVxpgr+NwbuT5X8nmuUud+FbMzfzbG3GCMaZKPoAEHAfgVAPi+/wiAVcaYXj7XcgCvBfCd+Twj3/eHfd//P5Akz1zP8tfGmOfz31ljzKX8HH5pjOnh7S/gVcN9xphLjDEPzqcODk8fXJ/YNXzf/73v+zaz6f8CWP44z9L1iX8AuD4xLxwP4Ps+4X8BtBtj+mfuZIzZbIzp5ufwiDHmh3wfP1H3+xouu9cQ2/TTmefZk/GMnAwxng3gPQAOBHAKgP183z8c1Ajez/t8DcDXfN8/BMD2Oc61L4C1vu8/C8AkgDfy9pt833+B7/vPBs2036mO6QDwIgDnALgVwKUAngXgEGPMc4wx3QA+BeAY3/efC+BPAD4MAMaYC4wxr2tSj/sBvIH3ORzAXpAB+6sA/g3NFaw+xwPzpcaYWJPy+aIFwJ/4OfwGgE2w9D0AZ/q+/xzsuXqSDq5PaOyqT7wTwM/nuH8L1yee2XB9QtCsTywDsE3ts523zYX9AXzD9/0DAUwDOMsYEwfwTQCv9n3/eQB2q7zDE8EzeTL0f77vD/i+XwKwAcCdvP0BAKv47xcBuIH/vnaOc23yff8+/vtedfzBxpjfGmMeAPA2UCO2uM0nXYIHAAz5vv+A7/t1AA/x8UeAZvC/M8bcB+A0UKOF7/uf8X3/1ib1+CJoZn4fqKP+BUDNGHMcgGHf9+9tcsy5AA4A8AIAnQA+Psd97gp1AD/iv68BcKQhO3HK9/3/4e1zPUeH3QvXJwhz9gljzMtBH6z59BXXJ57ZcH2C8GR+J7b5vv87/vsaAEfyuTf6vr+Jt1/3d5x/t+CZ7DOkRfTr6v86Fn5f+lw1AAn++yoAJ/i+f78x5nQAL2tyjL62vn4NwF2+7791vpXwfX8awNsBcmwD+ThsBPAWAK8zxrwGQBxA2hhzje/7J/u+P2DrY4z5HoDHdZZ+AnAiVM8suD6xiz5hjDkUxAq82vf9sfnWQ1fpCRzjsPvg+sTcfWIHgBXq9Mt525xV2MX/z0g8k5mh+eB/IVTmiU/g+BSAAWNMBDTjX+i1X2KM2QcAjDEtxpj95jrAGNNujLGpY94F4L9935/2ff9c3/eX+76/CnQfv/J9/2Q+pp9/G5Cz9N/ju+ABeBP/fRKAe3zfnwSQMcZYp7on8hwd9hws2j5hjFkJ4CYAp/i+v26edXZ94h8fi7ZPgEx3pxrCEQCm1MTp8bDSGPMi/vskAPcAeBTAGmPMKt7+lnnc+x6Ff/TJ0IcAfNgY81cA+wCYWuDxnwbwBwC/A/DIQg70fX8EwOkAruPr/w+ISpzLFnwggAeNMY+CvPw/OI9L/ZDp2QcAdAO4kK/RZ4zZDrI/f8oYs90Yk+ay240xS3V1+XcOwOGGnEGPBnABb38ngG8zLduChT9Hhz0Hi7ZPAPgMgC4A3zDk+Pwne4DrE4sai7lP3A5ildYD+DaAs+wB3LYbqsu/HwVwtjHmYZBP1BW+7xf42F8YY+4FkMEzrE/8Q6fjMOTlXvB93zfGnAjgrb7vH7+767UngTvI63zf32SMyfq+PyuSwRjT6vt+lv/+BIB+3/fn0wEd9jC4PrFruD6xuOD6xNwwxoQADAPoAzlX/9T3/YOb7Nfq+36W2ae1AB7zff/Sp7e2TxzPZJ+h+eB5AC7nlzMJ4B27uT57FIwxdwF4QDm9PR5ea4w5F9RetoBWMg7PTLg+MQdcn1iUcH1ibjwE4Du+71foET0u3m2MOQ1AFOTU/c2no3JPFv6hmSEHBwcHBwcHh13hH91nyMHBwcHBwcFhTrjJkIODg4ODg8OihpsMOTg4ODg4OCxquMmQg4ODg4ODw6KGmww5ODg4ODg4LGq4yZCDg4ODg4PDosb/B4eLlFHS9zI4AAAAAElFTkSuQmCC",
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
- "source": [
- "import matplotlib.pyplot as plt\n",
- "plt.figure(figsize=(10, 10))\n",
- "sample_idxs = np.random.choice(50000, size=25, replace=False)\n",
- "\n",
- "for img_id, img_name in enumerate(os.listdir(INFER_DATA_PATH)):\n",
- " plt.subplot(1, 3, img_id + 1)\n",
- " plt.xticks([])\n",
- " plt.yticks([])\n",
- " im = Image.open(os.path.join(INFER_DATA_PATH, img_name))\n",
- " plt.imshow(im, cmap=plt.cm.binary)\n",
- " plt.xlabel(\"Img name: \" + img_name)\n",
- "plt.show()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": false
- },
- "source": [
- "## 七、开始预测\n",
- "> 飞桨2.1 CTC Decoder 相关API正在迁移中,本节暂时使用简易版解码器。"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 19,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "WARNING: Detect dataset only contains single fileds, return format changed since Paddle 2.1. In Paddle <= 2.0, DataLoader add a list surround output data(e.g. return [data]), and in Paddle >= 2.1, DataLoader return the single filed directly (e.g. return data). For example, in following code: \n",
- "\n",
- "import numpy as np\n",
- "from paddle.io import DataLoader, Dataset\n",
- "\n",
- "class RandomDataset(Dataset):\n",
- " def __getitem__(self, idx):\n",
- " data = np.random.random((2, 3)).astype('float32')\n",
- "\n",
- " return data\n",
- "\n",
- " def __len__(self):\n",
- " return 10\n",
- "\n",
- "dataset = RandomDataset()\n",
- "loader = DataLoader(dataset, batch_size=1)\n",
- "data = next(loader())\n",
- "\n",
- "In Paddle <= 2.0, data is in format '[Tensor(shape=(1, 2, 3), dtype=float32)]', and in Paddle >= 2.1, data is in format 'Tensor(shape=(1, 2, 3), dtype=float32)'\n",
- "\n"
- ]
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Predict begin...\n",
- "step 1/1 [==============================] - 10ms/step\n",
- "Predict samples: 3\n",
- "文件名:9451.jpg,推理结果为:[3, 4, 6, 3]\n",
- "文件名:9452.jpg,推理结果为:[0, 3, 0, 0]\n",
- "文件名:9450.jpg,推理结果为:[8, 2, 0, 5]\n"
- ]
- }
- ],
- "source": [
- "# 编写简易版解码器\n",
- "def ctc_decode(text, blank=10):\n",
- " \"\"\"\n",
- " 简易CTC解码器\n",
- " :param text: 待解码数据\n",
- " :param blank: 分隔符索引值\n",
- " :return: 解码后数据\n",
- " \"\"\"\n",
- " result = []\n",
- " cache_idx = -1\n",
- " for char in text:\n",
- " if char != blank and char != cache_idx:\n",
- " result.append(char)\n",
- " cache_idx = char\n",
- " return result\n",
- "\n",
- "\n",
- "# 实例化推理模型\n",
- "model = paddle.Model(Net(is_infer=True), inputs=input_define)\n",
- "# 加载训练好的参数模型\n",
- "model.load(CHECKPOINT_PATH)\n",
- "# 设置运行环境\n",
- "model.prepare()\n",
- "\n",
- "# 加载预测Reader\n",
- "infer_reader = InferReader(INFER_DATA_PATH)\n",
- "img_names = infer_reader.get_names()\n",
- "results = model.predict(infer_reader, batch_size=BATCH_SIZE)\n",
- "index = 0\n",
- "for text_batch in results[0]:\n",
- " for prob in text_batch:\n",
- " out = ctc_decode(prob, blank=10)\n",
- " print(f\"文件名:{img_names[index]},推理结果为:{out}\")\n",
- " index += 1"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "py35-paddle1.2.0"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.7.4"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 1
-}
\ No newline at end of file
diff --git a/docs/practices/cv/image_ocr/images/image1.png b/docs/practices/cv/image_ocr/images/image1.png
deleted file mode 100644
index 8163e6d5df9f251b0c421cd9991ad6707fbd9297..0000000000000000000000000000000000000000
GIT binary patch
literal 0
HcmV?d00001
literal 133262
zcma&NV~{4n5;ppdZQHgzvt#bqwsv-G+qP}nwr$(C=kEC~ex7stiR{YgsE+KctctFB
zsxnMoRvZo*8yWxrz)4DoC;|YWiU0IbNbrC6?y`<`007p|LReT{QdpQ!-rmO4!pZ~y
zkO)gkflyZ(L-##NRz}t&1XdKX$(Mj5Bnw0r{8dg!O$0@nh>7eUj;EvA*Azji>Zl?z
zw}ckkL}+NHa9M3qms?*?k8V}k2?cf9UgK%gZFBv}<#0U7MZxki4ai;SOBu6Q&;i^R
zg59^?>MsW-Dx;qvSnF42=o^v;kK;6B!o(g(fv^<>v3TAH9ff`9eg1
zZ=v8cA~mR6YzCk=0*u)W2r!69*^;eFs8okp?}y6`>6RScu-K_Rgk`zSpPPU<60`{b
zqz#`;GyrC;{p?lq2w~Reyg}Dgs1qfCW|@3(2@xNHVo?ZVMkg}L_uXRga=T&g=|?Xk
zYphJaro4w^z4!(Ggf7{^q1uOv5KSD-yb35i^hoPg+uOqPs9PAiiDXiHrw(LrDUqni
z1Z>hxg+?nJVc+wF$!EC$Ri~I(0me$rXtoe9kad?nOA8a;@+K=^P!R*R2%iFilP;W
zH#n_NQR}dZREb|(d!P=o9FC`QH;e7gS{n>*7J+k10kIzUX~f@cDyu?{R?SweQ%%6;
zmHUge>sM1>tC9)JVLYuJPy|R1FfqCuH71w;(ee2)wC{OFzW47gLwHXb#N#)H`cHCz
zr&J!}MMR_xT<_ipfN|S1PNOGq1El*RMi-g#D9944+8uFs;}^s=rI9{-r5{M-D4Z*h
z03xVRpE@E4@-OIA*azC+3k~>UKMj3;N+1(`u795zFr|;P3ij*|l8cWEMA2(u3#1La
z()+j7o4b!?7G&F(A5>5R63>8eI26nMv(D^OOTY2KSTQahqf
zxN5G-DSjh7pP%lpnmoKymPQPaRP^CNS|*IhD7!jLTC|D5!&Tf$RGN|UTFpwN?*+A5
z_(~j)7!*73>3*GE?C4$;JD}{CnZA_Wi#r`xylsEIzMBKSS2N%&eJNY=S7g^fs6g6&
zRw1ao%-R6kP+REhV%dHi;YZ)R_6f@)$L=NN^q>Rw3!H_{+t;t#xY8N^xzL&Zydz1^X`KABPlB>#HR+Lk?k=L`Zu+Xr0Gmlz{H=kN4pIxtU
zQoZP2_DJWS3L7penoDdfYRqaRU#DBA;m)!aN60UkhdGr#TY5}*^muexhrNnh=lz@a
ztbHYQ#eFrgKI25m-OP=fHO4Kpa*c@?P?DGQSC*99~~xRE=l
zUn7_y7?anX7h({<=QtoWP!NnK6KOGW$mEbVkqVKzl131m26q690;k1W>AZSmU=tOL
z1BMkADHZjMrNi>BKgkTwHqE4Kpl##10?J{>jAg?-Wt=cZm2kZWKJXO`6PyAs!1&3b
zw759NI_+3|#=fAspv&UWV9S!W#@&!$x^*n`K=@$wuyi~!*+Gkm4up;zv4lQC_pOgr
z&snLh3RV%cWMpB9Wl+|*X}M;)W@ciOWUgY>G;?0fwv_%i%c5(&eXV_DxZgB$iZ%l~
zW5n6b%E*4w8tMS+9
zNr9DEQM4x0`DE!#b#`@I^@fg3%rXI=v*X?A@EpTd<|e3hlC|2_*|ov7!xqk_o-3zE
zC@;I)=A*G|^uy
z4)6|0x6fPK>(T4gvj_LEccl5`GZVsB)#ux{))x=3I?x%A7BDw39mt!XiXVc1Wk8@H
zkAJo;F4tNnU*<_)b>IE2aGy$Gw(ZPq
z+TL%Zz|hf9G$amJQ(mB&^c9w6kITzCy12&C*U_gqg#66>Lxre=Sv#YVDr^^ub#_f`
z9Rd(TSEdTX>*4j0&!g87yMhDLp{y{oLAkIUydQERP6LhpE0GWp6GMl1
zSw&4nr}>S;*u$hTJQT6tseF4v;gaE`MMO!7>51(mO)iF~gKQ`~3%mfeUeG{v(b0oXa%h1_S0f7}kI>bJ6QgaD1
zKa)vmat)siswPfhAVWv{)=+Boy7$C9#2yaM8=0w@SzKRjV{ISzqi$zzAw}==xE2%2
z4`mA_%@dg>WcAUWJD+TCeD@M>@L>6=h31ruz9BEFMFICbM_>KE{p0;E3-QHxnbjOR
z++9w=3z0)C*{DHiUz{j8RHB|IM#%ihPJA%GgohBmq*+p$=^iwjEx7F39N(VM{Air37PT}Q@At~>D$Z3KR8y+G
zyG+N^9ggsiGFB8}!pH`>?XkPv@Zq>RDY((`(rEq}#E#ILuCCtx
z+wP$Or&&;Ww*K!YI=>y;1=mF1r1iq|l5_>}k@WHNO7&H80YQwwN?=S-`L*_Z0vCcA
zB6Shlj^)JnwErF*oEZE(V1&TI@y>?-ZgR-L6Qhn_#%t;7xW;r|7c}x7gMoMRH=fNY
zXO#P{)=lc+Dgi}S_b=AYyfgLv;@W~iiCp$xCYSf}&t;rL_X&KCSx!yQu2Ww1Qp3}1
zO?D5y<%4BK$EjD{Ui?ha@H5lZ2t9-@za81(_*q3~1xB@~-k+Xq*RSKKMjSkjAdZ~v
z@vf)+mj0c0>zNMWRwZ|}7wb2^1rL=Tp6~9Ofr^9Euh@?&gnfcdZ%WtK%eeiyk?EkA
zvzUn-b>1@X^XEqiHT9Ju0rKy;H<{bWANw(y
z12rB!k523BsH?6Q@|)i~8}ly1H_lyVJpvCzZvzS476g(6H2hrNmY=w9Z?ZnRK0T2P
zCkVJ;jOuffG63CcP~U9e8Tgc4HiP{4X%2bBCq5_NULV7!F56sJpjsA4-$6HkNFU2M
zx%ywC1^o){atctO#VxN=%W&d_t>1qmz-@Q~UB13*CIN(Oz_*gyJ3B3PJwGU&1%@pE
z^KrEuHkBaA8*nXLmBnQ#upRU#zQ+p8p1*$MAPBdAKNtisL&lIPj;Q}FI;>38B~5?-
z2K+~d1b_lz1Hk@iK>vgv2d5{p|8vFulmEQ)f0w|y
z!2fRz(8OGj|5pc${--M0a?0<21(dCXh64bAPWB%Gl2jzU1ps~lBt-<3U4bsUEE34M
z>`yS#-VU=g)LtI>GS9M!Qjq21;uV;Z6nA$--_V=vPGAe;XW
zsLKOS!1S}#lKcG+044llp==7CCH?QPC4*uOm`@s9^34EE|L+hiYx-YC{b(d}u@b9E
ztHC$_huv9NsVX>DsQ(UaN#LnmM>0FM-T&dNpQ*g%-(1Q6j$J8abMPXXunHiK|6!Lk
zK=BXAZ{Yt9wfKJxtRtxn*ZM!~{%a9Uoc|Tq)_Bl#!856S%ijNP^*3+>29ZAm0=}cb
z$J$K$$IA^uz1Dd$_+G@dAtHG#+2Js^i?BR(j4Z6hhi%cj;I=4XX=&-L@$lo9iKWl)&rTj5o(p>V
z?BB9|KMi^8^x_c~|N6A+9rA1a*Lwvfg>I-Il+k?7=U9%T&x?zXSyO*J<9E_hU?0gh
z62fHtchYq}4@WBVbdK<69J6xGY`T3@oC*IXqDLr(x)VN-%KiL6WmUDame$5r`9B9U
zab02%@LebFMPdHa3fln)`vK-Uk43Ii^tPckD`2;)e-TLJBBYPuJj?@_gZ@J44d>QPD*Z)R=
z$V72uOtJRtT$(oQeSSYa7V>a%&VGJ7hBprVo{QlHhDv>nXq{ZwO;?L-Vf=1E{A|60=fN(u}Mli+b^q%pHJgNG535onmA1jXs~u4_et-psErzUm%V4cFVC)M
zMM1zWlH^Dr^pq?w5L#1r1%B1f$Y7gIeBtysbe-{t5~M1Ol|jrLbh=;XI|5x$eLget
zbqkBH9$};+@jR9xtQaaix~oT$-96vSnDi}maC)5Dd@FIu%4?o`n1~(>-pg;Fthc#2
zlS7p(VUT922Vhe0fRD{QXc4r>PL+=JW6!-jV~3VvdjeRT(3%lnvc
z6C8aI5;E$Nk47mxatBVa6^W%3|6;@zDC1L-LrdQ?S08u%H`KN+YI>HhF0V1SUe`7@
zF8IG+53cJxao-B}cKT@WRB#e8}^|7l#OP(6T1%E05fgH3br$}w1Jm>kY78W6-HL}M*r%17KRW!p&5LZwMxV^-jZ}bllJoYSoN#*4Nc9SB;h$pJC%hGem>AnC{(=TrgX*Ca=Qx5hIeQ
z_J};cy6~l<&E*d$R%DCG;=6bj$
z!b*jbd;Vo+ltJ7
zZfyutxj3_d(%qp9d!6{e?Jos^dpvlW6(_$u3!l_^-F&FHxa2s=wDmq@hkUZwZ3THg
zkzeXhvZW9tJ51GfWHv_aaAB^ZV+s-8IlP)OadF_?>9ygjktAO1Bb}reOiqe#PEa=U
zzLXlVx`$A@$Z*{j;RD9Xw+%I44*!}&60+^G!IF%lO_i^xnw8sFHo25w>%=fLdOS>;
zSc=0ivP%8zS$)p>a<31O<{t^gFtup!YM_OS=)l3A-leo*s+q~U&mokp@ikcGs5h33
zPM$*9{Bl8b1-^*r=kpVhZchg--GA+XHeGaxLQ%o=&a&PsF;`1)JQn3=dT^5n*8?yZ
zqlx|+k@)MRC<)?HO$5#s6H_4>(9&F9=K_5rGTVP4j#^^3<%2b(A!k)tee|zKAIdxg
z92iZhM?Cq*Aj0=O$Ots;A?Wx}3yOa&^4~`OUQ|0%f5d~~f|K5Y!;1eN>UlL`p-!^J
zOn^st=^**q^Smp0Gc4C0CMf1)(_=LLOTiS#WT9gFDkgr%EtvwBjDK+iAj=bdRC=57
zA5LJF6?H=nEz$uOe>Y)_lk6q({?2y6BH5|wh}k>^o`X9ci+X*HV3zBX9Rl+_6ZcxO
zGx2aqaPAn4+D>I$A_7GaAugyMU7xe{F&O)+RCWuTH9J(!%Lo_7KH(l
zakIZP)MOb*h^eI|M395`BGS;mPtFO_=)4loH%NqL+$ck#BNRm@t?67P*?5H+1ZL0o
z6?kcHrqr4>uZs{YafDx4coCJHuJSl|;%MJ_tMP8}c!=VVZoEF8v2o7feQ(v;c66_3
zXtFkQ=+H{Ak&To~f&iHxZ$noti(t@`pvQ=h@N{udE*|d;q%OJ~`^_dMLwX{eBxq44
z;1AjE)GNduTOoRG;0w%{O#+*oxrEV~at8#XbDoz{vgeDA;NUf>VMIw;L&;reVlb4{
zh9lYcsO{=)77iIMaVXNz$SYQ_md_h|
zfyiM@wD+8C537DpNlZZkE78pab7K4G6;~_~d8)V1C5WoPF?}rSkFyC6
zdIAu3yLosvy*lxbHCljOe)j_sNZ>;d>@?e1rHR2;l69ObP&S?~kw8hdcizz>D0ED`}+sUZqqe$=l`Bavw0wavMFzhfn
z++FY`Cp+sDzTE&pdl}HxT>P8>t9%K~!>7Vm)WTKY8zC!?C$(Kcza$TyR>~wza}aUT
z=1P?Zz&ng!+FJ&r1H$rZ$Wx6fjA77FuLe5zrC;cpB1*tjx9qVeZJiUu7gk>#pjZFJT
zwYmWp0k6K(v$}_<&*3+=oE+PO#Mg2zo7v?EjqT%k@w^}cKJU5r-9yY%^IIG&o^ZOb
zY>k?^XmX26msNKW4@6HD*W;9BiGb1fn?6K?F+~=EgTS#S%aX$6OZ_9}MBVuj25R~Y
zl+CTbp<$(dpG$r_pWM?&x--Ss^UbGoh+cb6&yROY%S)}6-ow#Kd8C2X1`l?|GMtr0
zqnf#}3QuiT+emszq21qNu@)i_QvPUj^qePNkfD-f6UMI@ldATRND9sE2Wv0eCQ38m
zRw6Sw;gB`p=4~P}QJHpw>jXlzaL?t(>cltD+apC^BhvwsiS&MA`tQ+FPjc;cs!wJw
zDXGCBz20QoIMrbwj--?!vR~OF75;-g@1~`80VuL^Htts=>-IVlubhwHig{+d$*#jO
z-toNFz=lq|y1J~aJ`^HL$UV`5+Y?N<++V!rt_l&Kc?*W1H
zZMTmsL8dlo^8}WW&h0*n+s25p1wu>kA`+}b*$+G@->RxAr;g&-t=)eAeeL-fF|sOf
z`q}3te_6JY@Ht<6XbsLyEzs-UvN~Am8F1Xa{RzLGdPU4eJ;pzo2#f5sZEP$}B~bt-&XuxgbPG3xWIkHM-)=ko5Qm1P^s`*GTUC$SuwZjuQ_P
z|BZIb7}qy?u#iX(NPk&hB%k7*-jL)V6u=Zd2~Asaba$f_G}BN7z>DqlRv$c$+Tiof;Nh0;Rj}J}3W^hmSiQDCm
zhwwqrh464(!2t*hf86z7Uy?+X0VQk(8vBm=X|WBw=0E1@G?Q6zjx2VXQkMAc)pv^C
zT-^5v1dsP?V{Ro3A8K$Yd$%arZpiDu!T<-h)8DAF{$rj(=Friv_~y|0iwwVBsxdLd
zLIqhq1KIpZy8Q)l5XLu7ReKs9|uJs
znqYl9K%L=9_*8j=ZVuD<*P{zjqrSPgxS0745$`JO^nUjklio8i@j{!d@8<(``kik%60M~j+9fII2dg`
z@Qh4wl~rO?EI$F}0R#_iU5LOuYL1z3Vt8545<#LcqTSUcydV-6J*}WBa!}4B0jiPNr_$gVfyG?7VW0Mfhv#>
zG(7p5B>1d?))xouLBsuq|MG|P?O2v_3Gb%5#FY_e@*8Y)6n(e&Yt*16`A%i)^EyD_
z^fU4E-s6Q>=AUN6j
z^sF1^z?4OtPp`TzsAVItZy~pvB%iG)A4jlGxX?+AZEnahq~|>khWsdymz`!H$jnjE
z4zfZR5~)C>Iud**5`i%#hAq{qxNCi#g#}Qlt8+~d?-_zq4ggK`kb4HnPq1zozUTh1Ioa?)}qpOI+%UV6R
zXG+Aw@dVzzkdZM}P*N<2liTdAs59cyIFNfGjQeTDSikvuQVlCf{7eQOb1*&U4eA>V96e#gJbguU>U$=1e1m_9s06VK)4>Y+8yHZ&h*lEIV3Re7aLc0@Fo?eZU}
z>HgDGNRYsa+kBfN^LN3uo%33xu{Wmrb{f0Tv;ecab3ICQhu1)o#9L+oe1T$&*Ern|
z5@7J%MahZ*JL&9
zF?5_!u25~+L0`2SNfR|X`JoVY#4+{RpkP^fCKD%k_(6FXE)s2`Gb#Dt>QSa+n#8lP
z?PMS%#!+VCDGul?1S8bvE5L_FJ&F%IA(dWLX&BIj&i|e&;2~JW)mx>-!6xfj^YNtj
z(Iogb_rSpxT+83fG{VQnzv#))6}y~jGn`S*#w=_5qaYxg>HHBQj!m)Y>zL2wi4`rg
z##Wo(1}P?l>wt^yZIBAhIb77Ty|3)wVhbzXQ;;QtU!op;+DBp|1?^IP@D=ibUBimd
zUscX`X;oQv<>qnkhU(gOf|+UBWXXvAFz?={G;+1-Tl4u)c`9RUb#w~HkAg@AohrI(
z3Ef31L1bJoD3tYQ&Yl5aW87YxvphXDk#@clU323a1sVq_kCygmT?yq6F)r=+ju|
zeWR%(TsSFwHC&F}@TrZtIYT-qW!I~L(p=q>SQ&R!rr&R!B|iS(P2x$j;)_-CU~f{s
zQf*=!gbXiE
z22Al4e3l4&=UmZ3nSLxj>Ov?e!7GYc?pUn|GdM1K{h{bP7zTn0eSqp6=)rkxeB;nX
zi1hB8+4R^(kU9Qd&oG+>)%>mM5htDq`KED0`%Ib4YN?+E^r4!gLX
z5ZW~o8tn-N5&*Fbjpxj-#`}gZZ
zNB`P5B_C4~2_>a`7>=XDOGHoZ<<`pQ~F#Aik=A=
z9c$3_pdzKWEo>Ug6hyIDueg_D-OkqL#|ef?oOkQy{(uCtGX=ru_U_U!?)t=<`);@I
zQs0*vK(5$@HSdXN`onZb-6qnf=VP!&7Cjzh!aYQ{B3plEn()d}8y4Tw#$UyK63^^<
zAa$lsKf!o36p&X_){tJCi={}?4^L2-f=snp@iNto>e@yy7qKTy@Qu503k(w|sd>JN
z(jR9t(Swmp3klF;ve~vVFmmB6VA4R3xH=3&7&-dH`~1Cx9-=z+RKky=@p-O;iF(^~
z8g)e^I0?0%E}ZZxeRjXAp_t2z?esi$*Iv{a&3R?f$(+cFUMx*d|w
z&^d;pY+UI15b%k1L^iY{G;d;;Hmq-ePPPf
zl9CxRwiZwLrAP8zCHF!E%Lez-q@DlvB_m@JGTsgX50sOf7mKI{>7F*%
zmg*Id;Y(>^X*Z$5h)|t&hYYbcCF|&~>(TuT^cuDp+`_drh?Wx_dQmpQZB$985T4jKqtw
zs?)wjvEEL&p;g2f=bxqM(;|pm5A#^uvGJDKG6i+@_YriIhLO{8~hqn
zz@CbEhrA_ZT>5sB~qe{@J
z^lc)25pd)p1%-sPUftUErS(=f6JkUZ2y-ce
zXwS;v$D>S2v8izWdct`!yE3;zQYBVMr!FJCotq2nDbv^pgH9v`)1F!sbsD^)$uC#s
zAQ3)>)4Gjbmfox7{|HiaQfoy&3lYmBAjmjX6A)eoP)q~=b$>?
zF$9vVs4i@XU|FX!YEHhVH#WWMqX}y!22f|y#*hQhgKn>B9sDXR}q|8L(~25m9?5
z_>#9bn+M8ePNOYC}r24Ul7^OEz6Rigx%h)*J+PK6S+
z>4kPh=O0e_oKNcXckyIprMz?MP`MIXU!+j5yHw>~2;7(Wv1N8E3(&u-)Bje0GU=S(
zh*qvJd)FK+``)Kf!B7`JJiqrlu~AiY^Y!!2^4pEY_^~BFFmyq@V?X7_3b1Q^Og%g~
z1$%f_4+Ke|G_LYFrnNa|2`WN7y=BWuQydCGVTw->BfiI;RkYO|0c9S=QAIK^V1@x?
zCy@N5&Ne?B>*cf$XI>VAl8YYoB#k$PRQ3YXnr(cq4E4w~r{J~Mh<}bbpJ)q5{7VI!
z2o8S-U2AQ_s4tcbdgAIe7B&nS0aWJrNa0*DC`;TOK2S~S#1FzVSK>v$?>W2G<5DBS
zvWKO`rOMSWOgwY=c$A_$4xR^F{2XLQ)ZGu_f0lz;hD`XGfr~7-UgCw!ea18)WA~*i
zEgqQsL_}V}weYNe@Ygb-!abs+jq5WVtX@n{^lMK3jagP^H0;GWjhZtI^{*e&8S#{{
ztCAIusRIRu&eBVyjV}g#0fFs%qPYFCA<66q`jlLnuE4-TY)$qrUcNA~-d4FZbDM?g
z$&NtC_c;8$>|!(n{I&SSO*8XlP(^jgZ&n!&>*87P`^=0z5q|}lyF<1~8{j-qN#F#P
zT3_Si6@pO{oyEay_K|eiK!ySr4fokTJ%DQaxNh?ruqY-vIsE2)Sxq_A@tyowQ5>+u
zaQb^iX(uH2f+1*u3J!(4Te^az5rv+D39ow9<8%1cuHbeVek@i>0YKvLfT3ih`?Oo0v_sZre1ae3F8*Zz_bG85J?nQEJI3^|rB^-=g!THs@uy-JbzQLyE5H4U!gPP|SHE%5sSA#im-F(P;Vo`{6k$*26w^W?`Fm0e;*|!^
zo_FuB!TCxd!yjp0o8NYa106^8^r1eFr*U`vU67-7Tq%I3h*QLd9vytC==X+?nE2k-
z1OJ9HT&3Q%If;srMR{J$p2fg(;;uEp8!etu1C!KsF)^TsW~F6(;}BXdS7WnV5fxvq
zt-h0=@YX(A6GeSVg|_N;-0I$WlG{6RseZZentg0vns!R_*W8E9e%B`Z$VFsPJK>i?
z`)N6csCSY60jp}a(Z>r1}D2B9Ri3WNK!g0+*ql21$vX9ci88RjJ<
zoW#(1OxH;~S0Mby$xPMnoQDq5`tmqP7@X*Gnpkt$5Xir>n+N!#TH;2v$8TgGG0PKB
zY}`J(cV$(lQjGdj5G6~vjYbfi`n5+!fPk2~VF+(LrCb+tvT!9H6nbu?4)~#hv=%;3
zmajJ05s2L#911p8?7rj#p16FcT3CU~!;OBgN3R#+sdz_LmHcL}RPetUjpv%2N9N!s
z^0Qb`N|U8X*2wF)Rl3!1i$x<*E4h9+o^Xe9vb2^ppwnS=h)1dK3|k=qVFZ_=l14q0EXrT3t0?GBF@J-amPOd
zWcjP+`r*6^72as%!D`+<`-G0*Dnbid;M<&PgC)4YO2R)1Zw;~A`nRwl`6saO8MT{2
zj=)oV2)7<5R&(08^e?c8{~^irZG<-ju5|L8(Io9+{0YKJq>LLZ49aizKSNMssFQU{
z8QtsiyPmNXh8odrjpe>3H2ihU0B%eIj7LY=8wQ5=%@r|Ks$W3}o>fXX!pgZf+-yV+
z#Iof!=-=Yl%~sV)LWPZ=L~AzU(ntx%?KiQ!d=Spi+z?cNyRor>H2HmeCf3qkqd1t_-HQ`7bC0Y+!}q8Dh1|z8!@_j@6s9>4~Dw}
zmN7Jlu9HZOJCZ6`N|#k1{CeBB&`-5-oY8fAigx0>{V^(165AvHCY0<~?
z4XGK@bAg)diIwCqm_@_G+-L^XVTe4t2xhalyEX1)BAS~B@XX_ef?s5uOhyQDpkbdP
z^)@E`DS~3=sB-*W(P4TFb~Ahmuc&GS#zy;t)$hL*
zj^gbgmvh^?F2n>c5+{BFpSQyQ$mtyCnqTOYby?T&t7p0%S8NYZAAwQq=eSw|+S|3>
zqVAmc2Ky!>o2X$(DMk%{_7PkjhOJxLp0nmu?cR&zXubM}MQ%i1l7|%Jvk5jkcD$Tl
zs^`LT5)xRjMjdT}nv)JrNh&TpIrUd>%o&TJ6}h@l*)K^IEeqb&IuHb2XI*n)L#VYd
zb1hphP{jhPV@PdOF7Cs@_dFqjrTeeDV>_+xWfRBr#AcKk77<3E%twlhq!}4#J^wJi
z$BuEl?_2~!^>OW4_SnjxDn4uf22DO#LP$;xAqz3UI;hc8
zLaFnYdl*)C*{_ihA4J2STuXxy>p_qy);4SJptWICP<@txNa{$nxUOLW1lXKKb{`jR
zmV_GxG4iCXy_WedUdkiO$a=+0pi;3;#%E7Hh&|E){>%rfO+MVLZME
z=arRU3w0qfxeKcCPXTS_o@5CpW!A&o1V*?I!F2BVa*&|2(PN;*tj{*l3eOEeV|aZ4
zd*6I2y!^tj&nfIkmSyFIF5sF;p7>AghLDr*xy7i;LF}
zQH+I3jg#}f)gW8^mlu*7O|VdssR}7yxkv_ICRP|JV@n#vTs!(*BGdr7&rFcqmpX~6
zE2rXFPG>k+QY
zt&}**(<6La=W{>hNmOOC)n9Dr*2HKRYX>)2Z%dE*y35?~qUI6HWzmt4;M*693C^2=
z^_dTX9WOe_zd0~k?^y|Y3GUv~SmhQ!C&Wsty=cc}djh0lvF9V>F}x_7Ybh4WA+@<@
z{<)#zy-&e0!SHlr&I?*ihv6J2{5Llb9(h2`d|>>7$=?(*_n)%eCDXpmN!TR%igJuk
zdFr4+hY!<2J9LKxE(n!SpY_rF3>06OJxCqrG!+aM(=WO{PUAA>t8pN`fAoBdFNzwo
z;FsVQqqxoJ&;}+V8ywn9;z^Ji@rMt<)3X}5b5h%FmB%@8m`=N9!`f%kYl#tBRre*o
z!$7E$ZIur`-LzwaQgp~`Wf?PNyu>aATul_Y3$Ebc&2)SPd|fxp@UwL@26o-wOAl9w
z4#uoDriozpsIxvW9P=a(XVO=307q4Iu;|F*GaNg3x{t2gQw9TDSqlL@pvDw4=58$z
z1fv(FN#7cSPSi~Pg_LE!i7!9L21ZKX^?-deG0K)j)?-&2L2!G1Virv)gTJWr6NKu8
zrYjZYHE=-F7J2awzAPxt2B1{QL^BR&2Fmu-aw&q_5-+lNQ9ROJpgKBZT3S&AlhK1T|1{EH5SCnab_~wTFT$|-dV^{Ahnm4pQM%Rk@-et*dP;agvp`w>RgftDda?I!%$x
z-%y*NOW_^~PC6@ja`&Bd_`lPw@YO%`;jms6#vM4{G9J}ylN3tZ*SsR
zv?%?6&p33&F6DTm^AxoPgz@e};(oNxm$mTujbHZL{hZTG7o6j!;)<+4$ER9n
zvS1E{Z9n$7^wo2jX2!ZJ*_S`Fn5py+q#$?_yXQm9f(rzxJz2wYnRFM
zYTM>A((6cQSMXTBSY#1FukE#&6vrs$@2V(doI|L-bPc`J%;uWmnXGOSthCdo4AoPT
z#f<4QT5xERw+!8M$KN3dTXDxtK%X~>^R@Tn%#1IX&KP%U76xPtG_<8`->4bERud%U
zV1T%+z)f@mrIA33MJ(*F#&*wy?UbXwN7wtiOTpU?XK6l$mUx|ug5M1$5j2eIipy7^
zDm2ZkE-+~>;BfV>2l;)s3LGn?XC+V86=E86Dw^@-iWGC&W@SIzt9I4E3}BFCH7es@
z9lY@^2TwAw0_b1fRorr7fRXd1HM@
zCPHFwj3ZCUgWp805$}^cMp};JKMX`J-6|EvvPW2QSQzO8(MK}}niY6{5qq+e`%-3P
zp$}7{`$d~iuZIV(8;Y<=wqEwzzH@8Z^iMmY{RFCf@4<)=|2i4;Q{UG|@u3uF0qwr2
zj7tCYbn~^SJHmD`<+eK
zXO*><3)$YO0InGix1%ryH$c}vri#$cgL8rXffz^Kb4FS1WS?F=n;hpNIUR}gCR~if
z38nm~GI1}RB>f3o-j8F$*~cYv8DCKo=dHrUndAB8dO{b?a}F&
zznBM~ymPbOw_FA=bRjxR+dk}GZudYi5F_~Ul9mhm`cFR^l&-~|Y$fLOtT15s&b)oz
z64%j;{Hl{m^;(&)aVJU#BMfoDOH`)j{tTcTOoScN8Mqq$y??zc=;Sx3bAMUrcsCyz
z&>#E6+v;ju^s3NWU#q_jdgVVEl1z`O8JBNv%X5-rP6fLMd+C*6wM$VRG}CjRGqAN$+4Ww7JD;R
z3PKfpW;a`K-_SNJU#^h(zX&_Wuu9`DUT533C)=2esYz2!*m2@y+iu6nHYeM*o4m7a
z?>fELxz5M)zTcnE>%X40?%%!4j>fexgCqti^fNq$a5Qz4&wF;<}jNraOArG2c
zFupP!%5c2IndPBU@|b`s7gijGLY$i=|8=%LhemrL+gCM)QIi)?zL95|72uVD_3`tk
zZUO4^KUDS`Q$ZNJT1iA|Wr9&CxDTcF;v?$?N}!{5)m+9Ttymx1UJd5gqw0aO}T
zgu}B_dyF66q4Mt9jV6%aUd`%$HtVi>dJJQPVqUNq%c>n$<~YbVu8IO5T8b18Y6o_|
zw^*b5G{;L1Ob5Y(^gDUHOWULD7txc~rlMuH#uD@$H~XJPmyJao04Zg?$dc!%Zy<
zNb9WR;Q+p%=&f1O7EPexkRWYFE0=w=880bUX#jiXqgDCA1DKL<9c_19Nw?Lb`d8Ca
z#X-9p5Z`F8u<{qULaUzm9yd=!m$KBwd@ef!;@e7m!P!Ss;b{Tb9!uE1^R#Mc&>rb|
zl>esRPIQ8apH#
z?{%k^&L9z*pG4pqiIzARP&zz}tfp}#)YBo$R$Ld*NtE}OAj3CO_b!wTr01ftgkanW
zJ0?izxCpxr0KMhGSnw?l$31H5!A;}3;+`QcQ@Cq-stnyO_@|9LM5-<}O{69Y7E=IPp0_ubxN^9+SG>saPL-Bo*&&Sfer}fT9{a4dH{6k>{#h+;f@)>>
zm=(7~g>r86vt`*rD)vYk*h)U6${BA5ONB{GR3<)7lNm6RUaV9jmXDxzj03y$m?9#S
zfWGoMlbAd6`H={!jd6o1I=QkPG{xkFT;7K&=uVYgUb*iP+xTG;?y<4hw9pt&zE~B*
z+Yv7Rs)XFyTzs~P!jy-f#`SB8qCD6-2ofn)x#l_(G&*VAA>DKdgxRO!4{Ps@Zw){?|~TcMR=S-^F((GaPe<
z-Adz}^U7p)@xA#5S!{*+PKH~c+YtknonK;a)3k0xG$v5o?ZM)9`~0v
zVg~Xcjcw)%XB`4_nsd>VlM7m4+^dddiD^ocVv0D?L_XJv^`^_6WIb!ViiC8>slJY;LxZ>T5$eYxr#F
z5%RM&t(BbX0WXLa$75y1d+V%=Vj8HpdJ5%Gg7RDNg*4O24)Pp+SRb}we;dT5hts_X
znuB$5kWF|LXslSsv8Z-6kHMvAxj7?0-YEL~e|%bK-P=lRw{PO*bupK?*TiLI$2#{9
z+9|)_K@csYpW<6expp$|yHRSUEqwoZOPGmyX8e6CaQ4!+1NPEbCLgqst@Kti~12BPqZ>JIU|+fHn|T2x_&S;x5h?3C%FrBV-NEr8S_*iy*~
z2or-5T5zo>m5tLNROn6W%fGk5Ip)j*b(*nSp!PgzZKDCC@1yDPMGh>my6bx1N>m${
z)k5Zz#3je=BeMsP^Y=Npkv&fQ{Q*dXm1F$8ay_{v9mGuf&089IhM;c!dm&xAAWVvl34dRYx*!~o%ZIS6+dFp1
zC4GOm2X51$)o?320rE7EP>hI~=Xm845iu#Dl!*HA{7?0{E?~iNI{Q
z7@~OYs0SDx6CV-Ng0;%6%*a)0lzw%+^5G?1
z1$ayXcqKh?Xw#*s*LhR`M!S223_0jM3V|8%d
zuH*V(q*SvH5N2*#T1??H?qBipFQ4MhDQOcAinNWbQT!yN+S_AhRRGXnRLMa4+oo7?
z(Ihh$e3efD=X|$2T1!V&goOdQN^3bQ>Lf$!;DrZWjDkg7a3_EXr>qhIq#>tW<6{UJ
zzmp2<@JDPwyt(vG#rl6PfaJU2JH#ZaYNT0rLt$2Uh-e>7G^;)njz752M&SLLEvq8S
zQiS{?YCT@jr$|y~|FTOpvwTNvn~Qq0hh+WNv?PcGsbr3iqx}=cL^+kn1}lOc
zG|8{ox&qWAOeG%2w>w|%nD|p-kZdtDeoTz!w@(q%w*;@SsonEt15)XXJCyk6Nb0gT
zImvEr3R`Po3SX*-PdHJ&28@5T`|YG_l0xhbgst~0#b2@9z_fp$Dq!cnb}>5<>8eu_XllM|
zJhj@m)3%OoC-*Z21P3i*&6nj!&sw;xD^&-ar*{a
zl@xcsyXQj?eU7gcM{RPchlx+$;LA1Dn&^#_JB`TBDR`~<(%H`M^7REG!S7xqehI&Q
zmeAW<&Mbmam8Og0qk@Y8_m6(^Kl)3bQ-aXOLz%hi%^hMTi2aczDb*^@D7^
zbu~58GI=cp@iZ8>1W&)U5|}*4Bt8s3dPF(2i|Z){`_j4tUH+O64idv2{q*&5K=qLi
z+K7B-XWpJF>X^a-hMH%^!`6LEM;+MaKH|7S6@tJfo$i5?Lro51t3;fq6Nid`Wp9s5
z?Lj$4_>PL}0ZX#px&eezy8Br`UM{wUqa*xNJUCFLD7_%F#&(}pW={f{f59I;0oqu+
z?}Q>s&_?_pdetBU?q2awQCm`QOtlnSHZo^+0tB__QQ=ALYEuJ*+}#zNHvy@?XO9sB
zCDPP>O5X`O0bgGYCK5~0FZAcBLOGM|N#pde;z#0aIDBA#efil-|AI-_te)M;
zwNI*__ehEyg}=)_@cPLCAS^iuF^)z%=RgqLY#d@6mfO(?8QX9*c_w9qP54`gw2J4
zYT=OY)EB-Xx`DE36$n@FORO?C@&*zhblQ9$5v5M3BlY|IEnep4C4GN@^RrG7^;1uf
z2%PDIo;U%(%Jn`tTKDqR-_j?Dy}@$kd`13?37mY(B1;7p{C-7#z1>nrqy-=R_*%!+ex
z$@WtHs;#oIw|nki&JJ4%;!3gk;7&v;PIj-f0ltgws%@H`-m9zsrcPa^FTcy=zs5ea
z7ewG|T6dHxoT3f+7`V%J)-$XdJB%ov_0@7^uqpan4~j$;c}HszXXxTJRc~;nVEPsH
zKlBOpa80Y9;INcWN5{%H=%Rmr>5$EYh<%sfi3re_$?wg)2P$|JVrt}GDzmh8C
z;_H5J2Zw)8aDu_V>oq3xNt-gY71P!#+f6yeJ&3qZhsXOdeTS8RjW*h4_>A+C?K|`)
zw#e1{pih{%vE=;`(B&9YkdsWG3I+7-toNe!1E0s?pT#~D%6j)BHKc9c)HOQWc$x_a~R)wQ?cv@cD%#($RuH`jU?1BXT+w3N-9
zq)ixm7rHHOhpsS8jNqy<`KwDartXT4iaBM^+)lKd52-_ZJ7!5?486=AiQ1LlP;>CB
z!2Q>yjX9cYL+%){r@-|PgHJuBP=<5YRMS%(JyMvyvd>Ss(*U@{d=On0Mgq>2F-5zl`XAn{gLYj;4cO{12$k?^JP|17J
z9$I{0bifs~$jkRj4d{mEdP;5yX@7YqYA;t{x3*@{-hpq*jCupP`{8bE&dRz
zT@!~KjH`1{2lxGHLEhEs8q2sr>&m0zJ0H?c9z40z@vroy!+@?}1!1Ai8RE>uSyeOZ
zH$kL9ADW9`|4%%I*WY%Sjr2EBT0yM+!r^fQr$Wl$wRbUZ+JdJ{V;c^T8xYxlc{}bu
z@Qv;i4?^vLMQwTmVK%cfu4q#G)}Pt{Domfa%MXQYdUr==Wi4nH(6B
zf%}wWDOOBOQ660Wo}-m9lH-6w`LtT`V?taAL35~Tidix*#F^FKHq8t$f|TtMrP*Jf
zldK&6zDaLde8Y?$V?O>jt?te<5x6ZN?Ss`cNvYYSW*2a2seW+rCg#Clo@kF)0iK?M~oz)1=e5N=PpdDA@}
z7o5|*3^(>RO0g!%It(w^)^Pdt^IMG~33Bpo*TM=?J8Fl%c*-l_=4~W*D~h)#tJ-pf
zF3U6j3o8)#R@!ymuo&lVD=0u|Rv!FC#xu_RHL3o1t;swNyZntcsAB%h9m21ho>&J*
z#~!UN>$34&rb2qS>aV9vH7XMg3Q}m(VXi7Xrl4UBB_)^~{K`5kA;lBPXrqI@Z&TH<
z!^a`tA$XCO-Y%K-J^abo^;H=WUbwBpm-D2^2JhzU7hXbX)sI_tkA`CO^%-|-yCf~$
z^Mi1NJo0wi9B})KUD@kSueC*i>V&b*hfKG7a}JKXO~CH1_m|VQue{uH!#}=Z*FCbZ&wtLC_wU_ObHi%t{`>dQZ7el91Yg%BEne>kt?+j2M*PqO
z+1BP#XmK_l`}eV#I2r$adJ%4;vHAGTFgp&m$|XH>dngMCh2W%6^lAS2PKmTAV0tXS
zQcY}JhlnUda6Rm%woM&N>X_PE44|V%K7(X4uLkJ_tM*_PF
zyufvTMth{BiL6nt2{wu_v8#0vzVe=1cJlKq_q7F&QqG|3uJU%(i&S#pJe2w7d(*)?
zYhWmg9`mvn;f(!WGqc>~579~~9PTqbs4ZTaQ7su+JX%b`-r~8jP+*?mXZ3Y=F5iuL
z<#kV{nqj9}8Ff#)Bk;6#Y0TN;I^I|}RWmR?CR}yS$C>u`a}U3G91J=@_%$F{Ehvl}
zLQ4qa(}(yCVPPNw9pCci&}9mOr~7a;aHwEMO{@-%n|%bP4hcg0nCdjVV*XQ30zW1l
z!9yApHXX?3&e;_T9(rnOBD>y#twcyOP9O8X5f_`AbDEe?N*MY+F@wsdOA3e7Z+1v4OvgQPFiV
zJX9{EG)rtHHX&!vdfz3HhxKIA!AKeT>m0)nQivcmi7X#iC%+(r(3S~4CvcNEQd
zk0FU0F9Q$d*%;2SkGsLNd{OD&sq>m+LcuYOzp`XGI+k(zF2!&4e*69flHEQ
zMPl{j!7IOD5qke_#0+;UA_MaFAE4U_VPG*DBp>2ulQrkP>UV<|GeBP~w}YH;H!_<+A7PF+QMKz
zvT2j?iQADusdKm@T)M*-^FMl)j?Hfz2<~Q-DDMQ}{gh`$yU{qK93nDajdl7NC&uRs}1$J%RRw
ze^K2}ZB(S?Gn&uHtTA&0dki~@cFDL^aG*(R2IK
z7u&YRIH}W(5W&dE?CL#0c>C|+@{@|hIJ_Ws3s2@OI%g;s#i`B@W~r%w1M%I>XgvR$
zcuf|zEt^Tg!0*fBTih{d+?*1zv6~cjqc4b@!7Qv0Sb>8oR~$))sVb6ufe~>QhF1{}
zDaNwd=VVx9_m;{b^}d+f!Q0@%5%D`lMLjF?B6$Zhx~*21O^4CL%uzL{eBe1*;lIG9
zIg2F%i{o+r7h7NCQg&xm#{CTFEoQL_im!bp`Sj9cstlL^Ca?FW*W+tU0Vdb)CQ)kN
z?n3n3(thOCO|)Ttsc-aD%1mG;za(jHQ)9}X
zp{R9{BdGK{*SqLM*Cfxx@i5ra+wiD;R4C`1Og#$SQ(}i<g-0Pyz0LpE$jD)P%R^
zu1rZ)2AFP!60V6+M)7U`$`&X#sfUMxJ_tzB;TlD2Mf6z%1`B_L>(GSdWroE^4*rB1
zk5F+RVP^imsY2mYMAJAaoxbExw;O@?ng0oo@;HT%fn2R6PB-pVX?>}SXJg+N#wLm`
zX_Ax&mHjM;^ZreG(^op+9q^3E7Ni(Pxngh!X~D`@N@kNkT8BoMNrEBT{>NmA*Oi73
z4lPAx0{?8k`pUum?>n{M^x^@u=P@K|W@inC>H!r0=5H5a_f}OfJ4PbVaUd(tyu
zk}0H`N(!}hYr9X{<em8ysof|G%2K|DUq(Dki??3ggP@3*I53OM+I4R^~B!4t3_UZE#X`
zmXC(Fpn{ECr+qfZl+5djiT`em@pR2#p=5M``i5itwDbeRI}CY5b{61}*~vu=t|5p{
zZhdwKuwn1D-gaTw>T!E;G-o%}=hW(&8F6SM^2nTZWPnFqb8-(rQm}_pmPJ!F6+*OrT&ryUN_c0&ssvZp)*%Fv
zNvokrJF@Jbj3^;f=#qApEFEA{^MkzjH)tQ
zHd82j9}?C%5G1zke#iDoly#xRMCt>({KQmnf1FNVw4}-o**OsXj~Kr{#r|G;V^6Vr
zYe$%M5IIcP+WcNg0-L~jR!3FT)c2QpV;IZmg&!7&-KT<&2YgCYtUvYZg}=8PU_j)U
z%cSMpjVFq>xhn0=^WD86rcHlMz*#e3Q<@SmO_pLh-HF#W!+tSlf5J`t4?|($0+I^P
zoybk!e3n1-p2Qi2UeJg9xZ$iTuV=W#zRW+g;FVVUK
zek#AjK71aatgt;G_pR!nS_K`d5nwWbX#Vxt2+dH6zGZ|pzrLFxEBeZqoYSie;^s(<
zV>)X%`o4X1K*^pHckKOe6i(z%&fq=G(T|cBg@P_h`NC9*)jDr!dH#CI~l(2AL^t5@4@Er$Lqth+2~P3c9#u>pDW)%TRxQ8vd9w
zn$tEA)Vy_icXP9Iw(ENS;U?+pii>vrv#A{O*!U_@gW{WWGO{F2o$#P`5^L<4C8S>=
za(Jh?k&>Y1JAS>9Nw5~{8IbI7{n3bDPQZLlxW>I%krsP;{LZ+GH;ipq>rT9~wki=R
z40*=hnH%D~!of38XhKJlH72R{NE?c3^J$)%)$Bar4mo`O4ng$Ny}Y}=&>m`e3kRQ@
zoyMWlm>>xQlR@AkSdK)HZpR}O?#}+n>>V#{XN&sg_9=JgE?2Ym_{vOJ{HJpY(#FXk
zEb&SNI{!9XX13*Pqa!CCPL6hOy&@;J$Zee~+D}#GqhoY>@?8b~mo*~%F(XC>5^(yv
zv+G69bYb=CernksgG03Hz3E2RJ6!=xqHiW)q=tOn%=Pn)G5B6O1!pc2YA$C$9qHHN
zmgL7vz}L=CQDC~SY)(HL=yUC#@uNr$?&+n2d-
zieC{*j3w8^IeM2UQvOf!D(Hd@_sL{v>(>ptf-nJ?%d$w|s_4bILgUQ*=Qu*~rmu}5W
z$|mbV!P6eWCE?Z2+Y2;sh-C;5N-YUnE|axQwoWicGd$xj+DJg&<2G~cY|r)jD-6?-9v+lG
zJT&(jFgkNi9RlLgXwe6dypo$GFL
zIu3gZ_*}k&FB1EwSvuq(LYTq+}!Lha{0
z+onS73AMsH?P9p1(k>J2)51_ah~}R;Ic_bm0mXl&0d(DUm9(TsDVFN$rZSGUF@&13(a77;l?N1Pd
z7hi964f34=_-xff_W;lPfT?l%sD){Czq_2a?x%5b=Z|;0cS$z@ToGS4ILlX=M0|S1
zeDU#XCi+Q7nQU(zfgz=)Jg0*(_D_*lt<0B5iCzP_<@jtutwDx5^m57-s3~qi+&*)z
zKU2D4h+1>IttimS1VkMEGlt2S{EA|qe{Awp_Tvyy%V7~I#2
zZj`4dgJ~#|oE_WRHr+R46^o3_P*+_oc!TJRDr2O3+{~`yZ92ZAf-w+M7kb<)EUJQw
ztz&AP!x42{W!mVb-zy|tX^OHXwNrPB4*k-3B#cq>&`=~v<)N*{l<#sQh{&7a6{FlS
zn={o`C@uYD?h|sdrQtITlNesdy~~})3Vx*aV90LpQ~&u?*emOu)9s?HPyl
zBeg0v#`gjVij+6}ZC^~Dm0g28y~Y+w|33FNX+d0d1i>iwB)$YAaye{ehaiXMIRkvp
znq@#u>`sCttD~L{2=c?_gd^BwkrZ=&nc0t#4~9-*i;l>g6{_Blftzr7Siv1QOj}w!
zL99c$_)=1im%;|H6`BPQj8%~(XH%wtxD`3IgC-^<-Yn9s1=x-%(MUK?9A=rry20U$zj!DOO_n%KnHCRZOS1pYMc$-j
zRNJ?=X~0_u>)q`2w51w2qqWH`Rt^Xs_7;KE
zUum!0*H7-U2>n40TQk4!mMV$P7hS!5QnUN-bJ}fGYIx8p+NT3x@?-4>@ZSDZmZ+XG
zA=B%L_1S&nh{JPtWF!lki#S*_8ZkQ?=GU8EgB)hv
za)Q1ro=^)Ep2xVC0!I;y4;Islo{kbCrTJbQ<3EHrJNyxHj*R6|9@veN$}!2edw~P}
zIbcIF;X;&rDnICNcD0Q@7`z0Z9fA&XHtR_hO8J$){u6dbo4(}xr8mtMCe!*~*`8ea
zEn?b$?c{VnI_c9YtBzSr^@qf_yjuGN6J}rsECO_n4&|3G)Zy_eXG-jf#c_ScgaeNc
z4{a)<&xQTNE0}oaR$G%iz$|&xwfs*xt%?sUu!57*8w#ZDG!aoRe%
zY~5(tpBe8+1qh64Kg?|0WaP&JW81=B(ZSkTJKmodWmD!&vF%Vs<_RkC8r*nQ(CN#Q(wY-q{_Lc_V&wI83_3d$gikNiiYOweG(-LN_(5DJ$KTPow9l4Kde{$5Oef6TewVv*!-on}Gwiv;7K1cdNigAzQJtAR13@KZ{nz_(6=tjhW&o
z!&U}74&%R2IdSH$@ap?;fZoYg8h?EekpPx+WH!0ds*AfVW1{Be7j?j9pqsk
zftd5gABq2bVY7K;;IqF5YVvlo;C3h8IRQ7jAGxv40%-|(zOxlRs_Z3<&`iJnY*42?xnYMeo5|}zEpz}^mySc?&PfG#8y^V{q@AIVr-E7VJtbBU`*{<cclMcSejaALjqPHQ
zoE?I__^TwFl?v#58yinh6qpV~%=MCe2Fumci>DO8N#yX3|Bd(53&h}zB$XE~SUV-9
zYZx^c%I@4xI+;5yKBUjah&fNt;237%&;8pTOB04m?!ZLytc(Ib~GdXb^1(f<5INk)uTD(;8j)f%=PCY>vC%MtoJlGTlCx8
z(S5mlNb^Qkc@_DJXJ71_diKgm=!N>V$KA_I`h|wzMHj!{4E3@6EC`FqX!5zJb!F;e
z;R33~CS!n7Y)ya+;*Z^LJ>f+mu_HG6yK(2Q1FukS*l7E(K8Dv0BmJ)KMTs=oXz2Yv
zR7Q-|oXH;$7PWGGA6kbf0p^6)!6r3`c3xh?%D!8>U#qoqpAg^sN(jsVMRs$K)?kZU
z(M6tCGD|{9ecY8K$E&%BQ5sg^X*&k7;SfTD_kI2gYEZLN<-K0dLK`FY
zq_&l4+ZT5)>k{2RvAYd3A!ENz>7qu&giR0sZQKG$z9pA!K+o*|++Qzn(IdP)KFk><
zU;F}EGRY
zw-~n6D+UU>!)Y{NM>C}U(`?R#(+(SHaD%|D9Q
z-)c0eC3Ob(Xa2^)X&$O!v&?fW*?6JRG<3Va?WR`TNxLF{>XX|oSA_V@rl
zwu^n-id}B_Jc1x?xnbH-R1*IjSwIf#JbsQF2@Sm=a{JCLzA^2t*>!__KD;(huN2L(
zkC1^K&r&a2SEcQbTTvGUDx!|;V@yTXi3d;)>M~WjCm0!Jl9dc%-vZin5_lfTS*nou
zbb%ykE@RJ1Me^r40pAWV%estI+CH9qA7d{&Gf;Jh+eW-{ZbpY42)9C`$tn_jj@fa_
z)=qMkKv@}XZI^zBULeKh3xehJ3uYaIwVCJJymdBycrO~!Intlp7=muhh76#z(jD|H
zK@!!n_3*U=R8^x$!)S$najJ|gNNP~^{0kourg&OvuoTemHj-{Pg|3dS-uZ$XTJVoA
zZElxQ7xW+7vY<{CQB<7A)^0iTC$r}o_VeGxv!YXD*%rJv+AhACIPw(TcPC#!)o(+7
zE1iXdP{61SzF>SftT6}5)y7|6ZP$tIF`vt{D-v+W{<={niH5pbVFx-dK^PZK$Y&D9
ziSlHW8O-}5>Kt6rSxZpH?wVW(6Oq-vz16FgNkrgW3C#Lk$1VF>%NV-XpWv>C4e24`vo{7?{OgEO%HLN|lXy@&
zqJ9+dC1f{Zh4M^cYoP>3vK+5Sk&UH7pQdA<)^oUgRYhLbu*u*5XTEtT)qWNpLHxdX
zPUj|g5b%}*Qt><4g^|6fF)F@@@3G1*oHw-*_(yArf2?#Wbor+6Sp9M3x0vOzv00Rq
zv`7r35_4XcE^(kbMy9K9brWZzK&H!KN4+)O(KQIPyQ{nw$ZXV#)>PdX`F>L;K^04m
zbeq|a_Ow#+*hc=A;kN{S1EVwVD0v=TeW#8DnrzMBH-Kk|@#H(|wD=}&1ck(s4M2(S
zSJ(UJZoXx&{bFxmTP@#XrIb^H#!@Z3cLKOOYs1A@P0{GRBI=JPYS9B|$ldd?U1~Hj
z`>y~@Em7>Dyy2Z#cJ~PusE2tKvGeyEwvW2cJ089=j;=b>QUQ*UV(#6!b||O!=gpfJ
z@Bg}Av`BsVZ(IgM*2$;6e@p{D
z+AqL4?PMt0LZT%B0kx>+x4$Ph^TA=+J#3`1dn!P*e|X;6&qJr5GT%u*aQmlcz1!B1
zpi~TemuUNO4Z`}i8y>A?cjnBuzk4KhSCH<@3h4Xw9)gm3yG149eYIpmCP>28N_?|*
zC0_UF8#J||(H-HNF_K|xN@6?o~z)?&4~dEfYn)SVqsJ`ziG2o%)5~Cr&;4*hfJedR4bM`^&gMA$LvI
zSB*HP1^*nKr#EU!L{o0c^(z4a2MnT~#?K*nblObK20>YZe5>WpgPJ9d_8f?%^!jI<
zFUKl=t6=cvhfhTM^G|7qqp!luEk!0&by@^TM=b<7#`zAiq}b4likwh?Sc#LkYqq;u
z+*F1NW6z;P`;z`ZR{3(-=Lh^c4RtjbP{zZfdQ*&yFo#*@^C=x=1YPJvUL#A>q%DwM
z9X-{&zHOL;hTT9I_JzIY+Z&Sl;%CVhn&%6vvuolwmd8i_f|%LQ(i?V`?9uI+ciDqC
zfNkUPKKV)4LwaL6&>yT9iZMy}{pxJ2miLB>RrP|Svg;#NSZ>R^V<`z_vt*n?mKhwS
z(+-rV%H$UApDqz5GyfX&QzpHHRuf^t)aMqjPhfy4iB}K1YpEbAbS&Iz=7=Lw~sv|dT?MmQH>1}}^Gds5tG{jo%Dl;FY8=PMEm
zP91u#Xn1nm1{z^>$2J8(3V%0%C5|>RYQSxatW?GOZ8pq<5|;FZ1pIl_VEbH6M0~Bs
z%6>z6B9}(elps=XG-0zwucP+Yp+S#B(-SFb^z7v*{^~4SkV@&58Js5yn!71Bwpi4j
zEX7?JfuThrT?WjS&?ZheTRL?`&JKYU4d
z=ubEcOVsK;!#wmlmQWIV;WF$p-o98MzA3e356gnQF2U-L{p9Z>`4&@L
zowx=OLT6AlH)$VPWQr-Y()A4ue;@oxACl&JrU7Iud%WJazwg4Nl;$s
z*a&ZRp0<^F(u9P4E%5(h%gl?t+xcm|+h-c$5D*|?$F{cbzl4s>grHIL-bo;GoSZV&
z4j{YlDB!>37ZckvLvYZ6k7PFtpzxo1Vp`^6JA#zc9=y6Ay%u>Rkcr~O97}}r+!TZ@
z%sf4dS@ZeNh>)`tNGdQZJlXiVdvy`beg0yg3V%=8+C|97tnl1nc@gV+jrG_V9PpF8
zElSPdU~4aF#eCFAwpDkEOEK^jxP1FXcH7?lDgQk9m-sTDnMj_7Q!8henB9#(I=hih#sD_k!dP71&nrFg$45Ov&>R
zl!H=gjmx-8E0jdC>Dscq-q-u$p&Xx=U=da_7}d0LlOCOU3d%}(8|;4`ELqLFL%v7s
zF5uzRZ(BYB0t^zFg}W^bT!bq6nR4$HPcx`O0eut>2TGNe3ix0;yp8~DxKw}5_#&;B
zd;D991LxW9GH4%0h8}p)j%D{k?E}aePWKRU%aW^U-vRoYK=xO)fwWAUiQG)jJUP1s94PpgnYaGX
zczwO8Kee3|<>++%T(t;Ha-b=CX9s&63CzNhWZ*Y=c>r(x;fc{PA@YA*_X9i5mVS`>A>km1j|FRDQhue+^8m
zDFaN){Tw=s=l3>~LcYaA%S_Z*{+Kb%r}05BzMAPH%4P`;>~;JpD3q8A5O9wK8TEkV
zj?thu#EmFpv)}?g5_8@cHUfW-q({`Z#~LhxKDF7Rc=u&ONo2zAGvGwvI4C97gt+Us
zz6MzfG>GSVKIhG}p$OPcW#FUfCS5q~XY2v`H${y*MzXf|Be|9V{XD?tpXkD0O6CTB
zp-+(Niztv(Vz><~>I$b*=$qWgW#OWzYdDXVC%G*=qc1N@?=ZATDF%L1*ZCk{A>|*a
z2@27dKK-)#z%ZO)*7d{1k=MJT2eVO~bRWsK)dc3Jkr4>A(7K*7Z;*grn;)T_49IvM<{;FC;
z?Cvu8a0}X<46d8Xp?7T@x^C`NT?Kf(t@Y0(a0l7D!gH5M_l%sB)R`V#^ip|01$f)z
z)+ln%eGap_0y;NP&tfg9>K)%n+||^)#Y1#398`v_8q+;c6{ViPpF~wSp4INsT)I9e
z2>XX;N2+-DyTg>WqLoSDh0O8!tiHW~YdXiG8I-q?Fo1?=^ZZL+MJTEie}n3Pk0?H`
zHt0eQ*LwJOIEv{0Y1LO-a>Q6tLf^Z7>Kz9CBjmTIsLBa^0V%p!-bX0fo+rZutsLox
zjzMNpw)8Q+3>&^JGdrD}Em?m1X-n<1)d|^#VAFQ~#9wN@XIOPA^Ke*Q$G~iO>!lpO
zLf3P!i1
zfbS#V?~E%uZ}jJ!-|WW7qApb$=C7_d?88Q0Fa2gj+;iI>oN);71^bSe&V~sDj8ArR
zAu+4Ph)dLiI}w|*stj_WLFhMKU8YzrS
z9r%N!m8Ib1R5kE$ZAO38F|QrIJ6}4z&)sHv0b^Mr>ZBF(r9!@pC8foMOT*#y7&~=r
zs%ZKhW)X__HX*>OK)y~1fn6*A{w@D|@psChO3SxIQbE|23e@IN@ssIhesm`0N2B_`
z=Hn@ai!5P>x>cc+FMBOmJ%C7kB!1Fil~2n9Ghw3hvpC;oM+G(6#x>DWg&1E?^}GYO
zx2^|~7>P&jhA7rHY2`Oh%HKycQ-yw_Tqwad2%K0x5on$D6SxEk*>QgI7I`^+r+em@
z2wBkjtwLl{mlE}6@TzA9cw*c9=7rd_^@^1F(sI7c*0OU|?Z|?E;=H{LWO)VZc$iYq
zzKt{}0LEfuk2ndVR)3Mgdq)5WfYf8-&%aAamkw6U67E-%rVu|iyFV~yty9zIREf{W
zeq};p3(8}5jYzGD(y^z@xtnEu($v2jHrmg}hsxG(py-bwf>6t4Ld)jdwwP?wQA)UHXANWAoaJdEn!JA6|0e|j)mkC>!fcghEp
zuCbOXN#8d=k4o8D+8!Q*sI!_{z3&|-tRup!H#_HA>BfaC(o#7=WEKQu0hI{3IkZA;
zLiwp-!HT%U)Q;_pX$IH6gXsH@0IT(auhD|>hP~Riqyb^5tS=2AySC|d4{%=mG1g82EFFa`
zgfB(jwxLoOd-r*zI=VaN`UmhmW~-PlQ5p6ka$jKO3QoA1D^mQ&*L}~Db*<8mf>Pko
zU@oDs-TZ!?AsChky|AzE{;qT%)BUR8VsxPd^G#Xh$8;9|-}CgaU6I(UhGvGF{`{oU
z=TpA&!9iyvAu2k)R&xcJ92^T>tn3QGre8torp2*kX=Nup@j?Gg)1HL&ytVaPeH%TW
zp7II?Zfj8__-p~ySyRB{^^=CV=x>GC-*I+P2^)9?LrV=Q5U%^Ku3M%2&eO)6WgE>N
zExSaQclf4Mi|YPhhNv9Wq;sI0G9dmYzCbu5MQlBv#$~a5+9Go3tBabFA0Ul!9hQYjC;QuHOkB=c{qDRxHRmrN
z-eCnb5&Bx+xqWZ-ys-t5G$q41VveY^yTH_ZHiGQYka;(sbK%xCH6kUW?6<98v_9V+
za>A7V!`NBI)Dd@S94k_+xLfg3+#QO$yIX04Z1V1h
zO*YxDlbOk6GV`A~=lq`Mp(l>;kP6K=f}9XtQ^x|FR&W%6q(b`Axj#Mwy(!>EA5Ajk
z6_nr-+zGR$P!i4pOktuXI>U<_LZ+Bt>J=H?kzQh3FS}?0T;g`Qk?F@!gZPUDMv@v+074oG3=--!%+qD1jC
z)kEc^8Lj@{7{bYkQ>0E@s?~Vs=9Y%<`t^g1zLPXGyen!nUkTCi{RDVL*0D;zBr`swvSDSNFRkyIK8Iptk>oUyeS1h%(s#_29l9rs7$so-#tXW4
ztzX5f2NPKp9rs#BeCIV%O!Al
z5J{|qrCP38;XNjz8H8;lkHM5iFqAEi4>+0meRG;kh6w)|Y52p_2?(PwlC#RSY9>#(
zkWK3=Z%WW8F9c2Vz&~@=0&kXK;({_Dq;s7@EOnA#WG|DfjZVgL7T~Id*3|jXzQ`ez
z0IswV_+DYFu7-8y73ZtbLCt*I^6m8aQZlb?7a!tP3#sAu(uPM(QeUY6q5BJEq=w1DEHX5#9E&zv>6Z>&}zLF
z>9l0p`N#w}F>y3&uQ+)2ATuyq{c>MN(W5`mfu2h1IOpxbjq1#50q!SeEr18f!bgwJ
z2lz1Vsc^Y=o`P9laa~}pGHFgrHn+JCC0+){+ua4{s$`u+F44M2e>4sFdM#>FQgv@?V7(xhDa+J8(26njgl8zVp(mh{N0C<5cYasxI6w=`IbNoK4`lw
z*d5RAgft)Ty3nQ#f7{p7{XN$?rl
zU9DFr!5s&2P0%a&NUGV19o6-~uy@c7W*ER~a@J}1m^`s_X0lcfDkZ@7ZEv+1PhtRm`vW4pLBnuy`IvbBSM$^@jK{H)OwQ
z+Ug6Pqa@qtfNKki8G79I;)xhY6-Zm%6H}Qn2EtNck_ts(54Iq6p%@XyrA8`|qQ3@z
zL&@sHK6(QzLXme8Ehcvdp99<55MCoXq-1aO4IyKI>pop4Lg(t3h1bBEZu_=~nMykI
z(WvM0t5ka9Qp82|<3rrjdt*k1o-=_)`?d1@gUb3yVdAH2cNqCn>
z^W(EtgJId{G_|M?woMo5>?0!#Aa7A2<09{mtO3Bg&cWcaQ-16Rm9S@4?Jy-H^)^W2b?iZ<{w}?!5gJQU)d`^%3K|+3edhVx2k>>z$O^$wNc)Znd~-;eTp$mb;;*QXiktrMf2%TQv0EXkRjEdHuiBRDe4rKZ=%A`0)$jg+Ftkr1>C+_SZ2kr9;f#=
zXt~R&&6hq8)nFfiJdiY6jE(3hRv2IAz+g@T(FO*g*ySSfQ8l_
zI1?m}$)YJ4$y|#f1AOA2>=I+od4C&Yf!YXE64_aOTnT2OiEW0-~
z`e$J#i&Lw8xzuN}E}qdf&@gRIFC#&D3q0KEJN^wVjWg|VsO!gz4(l!BkdjPC;1Ym-u!vt397U7j$2s__%u4zNOri;TNL&m`1R?ZD
zqjyyzx_HLO06w~;Wc~&<_@s^*Bsq>B0lJMyLkO2>;!3R6vWMRcm2#8)2?m+_9loZ#t-2-@u$J-QXO%g^Aoc<979MdN!k+Xd9dI4bhf+wK4?uDZ@GUL)m+Hh|aB*zYuQkK|b{liZIM
zWi$BG09E$f@=UOU7RMop>`5Lf7w7liSiGr`A;xQpbOb*wugAhH>ueT~h1>*Cr%>$x
zj%^_!A;&X&a@Q1XUsPn7XC)ZPiPNeb3~o`~>9(|xj2q7`L#^_@i}111;;^8!2$Ndd
zw|>vor7V()jsv&fs&GxzLwySfCI*ee@ZAd}I*Dh2qG_b5OC==)iW<5dCak}hj@X8x
z$Wpz{&Y=9rezW0kMBhFeZt(e1c
zi;dX-W0(IN(^cCo;-Kus1vr;G`hT4DtZx(69@mm!U1GBN&sqOKe)1>Ng~+g2DQr%)7LOVSU67UCbO#D`ymweTr6xR
z^*REJ7TcjssFj!&&sy!^l+jXQGJ_Y(GI|(+48dcUJ>s7>)QhoIRuXkaZS@Ck_q%pj
zK#q7~Duy^|r_nVO_n^`Z&Anu&AlfX+r3JdbQEC&}QXEazhbx(VB*B}8s3U|<;gOI?
zGnL^QCs7CicMn8FyK9im%YnYZ^$+Kb^>Ud4p>ZUkwg*bFlh;k}J&j;g8cyaifBLjoWL{||P{Y+5%3T4V3gEhqr8b=%~Nje}J!1E6^g$VY~Pa`DXh*W8*$it7EZ^3+2bF*9yEYoDD_d
zQDM7?fEJ$IsIhG^0(!qjptR*_KHkW(A28l2Yen%g1I@B?$Hwfq@_PW!V|L2nhaj)t
zZxk&n6Z2i>{g)g8!q%)L&-uO*U$o*Y^;qlXT5#2tZ>4#_MoV^xUMB
zEG~AtUu`7(Y0pQ=l+ERRKQ`ZC7egjoRJ-YoU0Efp5d>_p%rest%O5x;J9zjfD5G{5
zCH3_Dukg%_8on#{J&mV3qV$c2SP4aNMKJM=kgr$I^=n$Zn0i*6qR;5E$HXe^7D(6R
zG0&g{z9GshvjSF%dp_OlOn*&ksorAo=}V)fq+*WQ>>_-;CyH!v=O-qpqLm|
zEADe(@9dXin1{R|x9C&&JgMMX#6;I+w)WsYkgiwI>~)cD?kf#^X?%y;;!$@!xFQR9
z?vjF;(6~B)x|~Sqk@^8u7kN>JHtw9iP?7$S#N`94-|np8d$YjVvC<(|L}Te1(9gO?
zm~j1BSb8VFWxVDwPbZGKAS6U$I3tl;rzAA|7^CA$WMvQqZ>6-KQeo$u)*~yyF^?X#
z1Muw8T@YgL$REp$v4q>X!-Y56-_o|JFe5E1c0>z#PswiDXz_~798VM8YK^=w*boKp
z$=^P^@!~Kho0HGRolT&7`X0varUPj>w^pBSx@~+?rF@#!
zj$0msJ${%SlxGR>Mob#I9n|nypDRW9dw>?+E21>k?k$vekW^|jp8xS10qCx<{CSPubhxS
z>jFgb4SvR;gD?5#e-c%Ee@@;IgohV|?o`U!`1Z*F%`Z!0V>!w9^I6KoG`bV+#+ee(
z-RUODCvWQD@oR1Ep#^VDM^^k^F{T}WRZqtlpuK&!|PbzLy8qxfi=#r&7KCRA?hi
z$;kNZMg>7RB3YkoL$DlHF#fK`O}Pr0fKa)mLBHD;ae^s9k8`C2T;c!lIU=Ppo051%
zUd2{g1ZY(;A>_-0*pNHj9k#qb*JdHY43|s69|fGUV5ltgl{X!_4#Ga|t(y<8sEoYZ
zhyG4U1CuEhJnYeh(R6g&+?J5M6NjcBX1H2{??va=0hC7iAvCD3b9GtYdbueoKs4++
zdmuOs>yw7qi^QWiIPOQ!a3J#G^IKT?k(8~y6Xwat-RQagTRkT=@FkZ@J&N;SzOAax
zTkJZXuf^z!lC!wEW-ouVxA=<--sB*G(d@U$ju4WIYI_~;%gqj;Z=r($@FX{(_1#HU
zm;zxt{~A08x9?|@7WR1Wysz=-qRXI2_e4S(-ML0{Y8s^6L~JAJxT%mJ#7%Ut5s)y^
zgqW>6{hrDMjEHDlO67Nv2JXbTeZlW=&TZVlY0B&T1Mx+8s`K9Wh~26jf$7;HOVG%$
z6g@TQp{$4&1zN`DsV`S`JABE_G{!_A&sL%D{Ta@(!pQ2b^u=ML#i0H9_>c3UCnr{-
z5(>Y^#cJt}XcHndHiRbQx9UwHvk!eDPfzOOxxn*wpax)d1;`e*EWnV&(Ik?{5IuF|ia&w~j^iAEy*
z-${>BSYGgALRz1I$9D5jwrCxJd_|v`zcGj&9u!^xMri3i*{+wnqOvtW`=_bd{3+*5
z9b~T7;$#nOlqbLKV3hjB@aIU73)|(nVO4VzLeC7MmAD|5CZnXqUP6L=*mlnF4EWZ+Hb4D8kXQp
zI^I1q$q8)mI7tZrhG$-)G$-5N=LLOe`bDbjfNMo@%R!pJ(CZ+1-Yf}PUx!$n^1JF6Bq-2MI|Dq`*lU{0AfVR{(Dm#aQ8+TV)q+uAo?X!DwER!IpPd~nNR_}ADO#H1tR$6bqF^1FUkGJ!nQ
zBj9W!y~=xilO_l4SDrr^|5`iUb&u*0T;1+NkeLtp)`BoNijhWC)-+i)Xq=r|^J#e9
z?T=jbr}=YakU_0%&I%%SW*Rsj;_BV10>1626G&7pxZ0Tr{
zBBsEesLA#ir>N513u6zHpgL9zJE-H$rdTs%m`M=zJD);
z+yi)kkAOQoBghZ8b4Y18H!o*Ve>efXnsrk|S26H(t+n#GC;MIE?nsH|pc!c0;bn2h
zNdRBdhJa40B@8?(o@d@@o0Ii99@{#OGV(0tE4e~vIp^YLO~DbvfcLE*1~{7zP&Rfh
zKT+>w{8
z2O7&F6Pbm84O@WiUVHz9=kkVu?5W53GQWvUR$^gaNXe8C%EiT9DbfdbA#F(BMf*i)-d}F}8gUi0wI2zsG&!{L3VY^Zjx=aI
z&1*~2K3z^um$5!mozBhOVm%W?@j`w+dGB;b
zA0uqMbP$IdMVFtS{LV{XbO?!C@VpW5eir$7pQedt3hxeO}Kb3wv4qw+=FWS^zU*Dgz=e;jFur59pnTn3PaR)8-%68m~x28+T
z&HTpO+QzRjTv(AQcv-oiNta-zKP%-hTE|^ySkPvgo8*_6w)Z`w4x=!b
z$e&v9ZCO#8__62->1z52q{JN#IjrTFk@byJWFrt{^S{C#G563vYS0{}QSw{wE~zR!
zLf6#&^^h5Z;dh}UO0y!1V&+*G0E
z1;|L^W4N`cRk}80;p8jTu2-1r#x7}P3VLe-?foDug`e}NC2*mVz6RQYXroK06c8DH0L0QONNcs_uRbHHla%fO_cra`+ibzrQR
zZQ!MVvAcZ(pPM2wN|_DTLLEy(2``n>W}KtP-2m{e1iz@M&O|l}&TN*1mnkbiRb>&6
zY$5Jh2l`ah`=fl2Ef%%qeH-hybOu>FLM
z&jIy*L);^#Kn^J0mzPkkqAXE6)340ptWCcL=IHPeQ!rRX@JsN+{aJoQJh>rIJ#DhC
z2e_omz~=MSMgy5%biS*b+spf-5Nn;K9IBBSt0AGC>p|`m
zMI{@J8X9Xow5#Y*{TZRWngYb}4ad{eaUvO0i9og~aRlhp+RnG8L~4p_{1mxs#+)ax4)
z?|pVBh%HoqTy5EoiA+_*C`oj3kZ5=U+%pg&*?^@!j-I
zNMLuk9879W8a+Dp5fRP*7{}HcY$c_fx0!V)Ae#zAaDen>P%YVwY@!)WV6WVm#W%JwTRz&F7BtetuRs%(;YB><;YI>s0^<~qL>SRGui3u?W3uwsWKwl6
zY)gi-N1;UA4vq1w!fUIFJ4nK(sad5gZ&`TEwmnq?LrtI!%5OwnoA0yPZ+X8hr)KEf
z5Y0x&saxTu4+PO}X!OUuBTXWKq4jb}om`H8(m{K&$i|K)bh=7m&T$r-`AAc|E(HMs
z6!YXHovhg>flo-Zz^-ys^w@F$L9gScscxpldzAzb-rRUH%T
zLLSJds{l8Kw$7%_27hJ#tg@FSCQ!tlqL|D^naAa2DOd&JTEaS9vrMU#1%wudN3O$1
zpJ}eY(}r}~vHqL59x{Tm!ecd1iEM?i&!eK<=;!O=n+X!~U!?iu1GW61^Z)48m73((
zwE#makfZ*7_cpL1_1xjsjMA35f1s^!wpJNfjAtBlYQSeOg-!c?&xeoP_LlbLB9p}z
z{ZA`nsfZCO*K;NUK@-TpGGrAkMo?OLw;-EY8C_-RLBOU(N&)N65uzz=
ziEhZ+{zDq^b#K=(?LJNA1}^MOG}cC1t)mZngNlWW|{7j*N&s
zrLOoHXOHk+P^D%b`#ZpDs{7|4RD1(i{)j22)vC(@SF^@6&=}OPTgxyIepq>q%&m67
z-YG^6>60P2nqy|ml#fA`10DbVAiBcj3os-y5Oek?#l=_>}#-Skc(`EK_8tyLg28iUFWG
zVsWK+Wj(_uJOZ1n^Vdabg*t0=D;bi-xb4jyt=8(`8RntF{Gc^uw|lbmGzK}$?E%Lv
z_A$Joqee#=P8p;@We!Zwex)rjD356zmP}GwCNk4>>iSpaPQPbQf0i;eVGc1;P+Ank
z?RRX8<3e%I%Aq93tATPAbN|kH_Z9PNRG6E(CP)VLv!Dr)XSa2Z9|8SRYt1ePS(nj$XgKb4p2Z`qXNsoWuE|2nPZRX8KH|Pu
ze1QXKzg~c5w%-MS1!}N%T5!K|2PmXI+(aV`*oX6n5bcKnjgW}3yY8j6%1n~p-e$@W
z%8^q|uggmlag(sEME7BL@S8=#mxpmz)X`6$2!}jl8VM|HoMs=fn6<5+eH`@FOP{1oz
zqvQQa2Zc2fBjBmPb&SnO=G3`GYa6+ctNu8m@X2OJ}q^oBHAJ)xQ
ztaEZ>>?&3C?V@RFF>=?$bfz5rkE9+DG$QW)R5-Gl)RR>JA{4BqprbS%D0g;rB%4y5
z2MacQGRN^vfFMIicbCm{ldb{>8`}ok6CH6TIK#n$j(hj|)&FybH1Alh|uB!f5=m1wB?w_8O^aILmyY^GJUM
zUBPToo;s2)!PI-I-RWX=K&R+G7R+F659WAM#6sM&^?$-
zcAF&$yF;v^Xvr2Ee(}q5JOUCOU7ij-22_h2z8)W39VPAh*5gXaJv;k!pL?;_t@Pk$
zbCF*?y$D3A+HQ5KW=Q^(g9W*drX@QgS`t&nTmPm4cz(881|3)sWh%x?VCU5JZ)3ny
zUo$F_G_~epVj%(E+j&0KjDBZU?E(4AA0J+Jmu~2m-YB!pJGC
zmv2v95zs~!XYqt8DW%E4IVLd|Xrnm=mRLVsMY~iCm{btpgP7nAjy7)
z?T}J|k=?q*zXp{Z@2`vf!hfZ{NPXhwcB^X~N=6-yMx|djwf(@f3I`c_S^>SmgAX)Z
z00rF0w&GH!-HN2|0L78FQ}C`FtQhxh0(`c+=P!lrmd#WF<@G`Si62)QJx&5dH9_`M
z2%8l!F}W<$H6|nV`C~M%Jw)jcR#fpR?^1uP$dxI^OAG{a@4ojafzbvLTDL$<@IA|u
zj4xQs9-_~uE6;OK)B~84Cy?4IP~-Jm;+1+o4yI=F`iC#&Gy=1n2Gsiyy-CNM-ykAL
z?OqxjMbwNgF<1`d%N~RjPu14ZbzzA(bn7^4fz4I@DY~DR_9OLCtw0LD<=A2c8BQ3S
z##St@#HGxC?1rG?l3sRh0WWXxEZnY0cM~|~#wy?6Y+0c8Q?+L=6hsWjNYgPlzPmEE
zduSnR)l;bKm}>a`(j@XiJPMjH-{i>bMnFFfr}yiM>m4k^qMaPl5b+j*ZrqsZ|I9@f
zO$TOug4_mbQ7s_634}?)!y^J$+8{@hn#=;(Qo#q68zsKC&nNSF5LQlTvF?nR3$cGt
z37Sk2Ro#)F&~{Dn(21Xf{kA->(#wTis6jXnyK@A_rzE`JwUGJ#&cOlDd|6H;d=mge
zEd9YoaAs`9bfq}BlFfNx-5ZXYG4^Y4Ad?Eugqj_?^eK|AHVbjP`)RS_c4pqI3Ws*p
zc8?=xI7aLBKs6V#SDFpZt58%W?2K41YoR_2TVw5w{y;Yn%hOc}fc#Cu<*7G9J=2Pm
zER_l6=3os`vWXPUwX;0)5_8s&qeJ@+~sl6IRaiTiLeJy+^BvD_k
zLmtF77%5}f{K>wY`vx0!;hdo;XR$IfI(Gzr`+fbxo&K!y(}2_5pS~FnRg^%$3#un%
z1xBceEIU_pIQq-iU>4AKCCn^eo38?fcPiKzeDR`vMp#3|&9I>pB6S=)qKRwQk3L#^
zyrd+!vYt{qUqL*iO?^R+88MF7vU+(2+#|h8F#*rDBbJ|P(Jtx`h{X*hMB;@X+02ah
zk_Aq0dY-!Gz8KomMy{5?x1aQv>GKGkH-Nn&OP4l_a#hZHovMT7u_Wt`xrOqRM)G*r
z>5cS17U}IM*dQV_F0IjJqvFEz0UVQQBggB&G*$&Kn)!vl)N8HHwpMw9BPFETnp3Tj
zcQM;2GFFcbs7#|lg=X3?RzeIKZju;;o#tzirsHh
zKF;f7a4_K{vLnQy>%n$T#4Gk1uhw<<93lQ(g9;Kw)Q1wUzd~_uzf$gK({EU82Rlkl
zSQ8LIp@7p-$DVQ+@abAl()@ecnFcp;9Ww%Ah#(xHXG
zV>dUGMpiV)geXC(*64N6(O$s*7~W~KMx)srs)G?v
z{od#yV)GM`KmahR@(JT`24P1e7VuL&$$@Z_w}Gui7)fr(^_=s4g38x`-&j*?WrxeX
ze<-K-LgcqN>6ue6|Hx1L;IA|r{hs9{;?6V8knjQ&oG=;5Vdm^001Cf?@NMf!_EFC0
zQl|P1XMgy`gi(qbPX9Mukyld~&5Y75%3lU;V+;i#F?UdkA0O+VTm9!v5|u|FA=z){
zFZ8Jjn+?qoD!jGKKak^0*nt0<)sM4b!zO=bh|HJ-Zw4TUX+s5OP7^tuO^e^WcB3j8I*4m=(`
z*tcK}@Cu6`CKZBWYFGRG2Ey{V;BZ@swft3gM|z=b>Vyh)u)CbZ+=dYIOQ?HEW#hMd
z!UT`A&g%_26l~sy0d>{_+^se)4he{vI=V@lgNBrEzQ~p2Mca90N=zJi-|vuUmB}io
zxFm?9NH#?e#pmMDj>1ku+6T<*tAr4FuFVb4g?tHWPb
zHsyk8V}_aUzMx3ZDhkm0fr=k_LHhXrPF)ya?x0CNIyg7VHEU}IQCweWpB3zYd0B^a
z_I2CV%aUetJaF&O5eSpqvg8D^_jyrNrf~#c`I4+###2s}`2d^c$H_kEklpAbwhM^_6{NTmkZ%YeEl^lZRTP7@89LFHR$gm;
z&E^Q?7`cbLm|mPg-DbA<``2kckQ{12LZ+DPX&(shl)DKM)22<9j&nkAx_*6lx0Zc;
zT`Z^+{XglZ|3R0?KEeB<^Ct?4v