8989
9090 设置PyReader对象的数据源
9191################################
92+
9293PyReader对象通过 :code: `decorate_paddle_reader() ` 或 :code: `decorate_tensor_provider() ` 方法设置其数据源。 :code: `decorate_paddle_reader() ` 和 :code: `decorate_tensor_provider() ` 均接收Python生成器 :code: `generator ` 作为参数, :code: `generator ` 内部每次通过yield的方式生成一个batch的数据。
9394
9495:code: `decorate_paddle_reader() ` 和 :code: `decorate_tensor_provider() ` 方法的区别在于:
9596
96- - :code: `decorate_paddle_reader() ` 的 :code: `generator ` 应返回Numpy Array类型的数据, 而 :code: `decorate_tensor_provider() ` 的 :code: `generator ` 应返回LoDTensor类型的数据 。
97+ - :code: `decorate_paddle_reader() ` 要求 :code: `generator ` 返回的数据格式为[(img_1, label_1), (img_2, label_2), ..., (img_n, label_n)],其中img_i和label_i均为每个样本的Numpy Array类型数据,n为batch size。 而 :code: `decorate_tensor_provider() ` 要求 :code: `generator ` 返回的数据的数据格式为[batched_imgs, batched_labels],其中batched_imgs和batched_labels为batch级的Numpy Array或LoDTensor类型数据 。
9798
98- - :code: `decorate_tensor_provider() ` 要求 :code: `generator ` 返回的LoDTensor的数据类型 、尺寸必须与配置py_reader时指定的dtypes、shapes参数相同,而 :code: `decorate_paddle_reader() ` 不要求数据类型和尺寸的严格一致,其内部会完成数据类型和尺寸的转换。
99+ - :code: `decorate_tensor_provider() ` 要求 :code: `generator ` 返回的数据类型 、尺寸必须与配置py_reader时指定的dtypes、shapes参数相同,而 :code: `decorate_paddle_reader() ` 不要求数据类型和尺寸的严格一致,其内部会完成数据类型和尺寸的转换。
99100
100101具体方式为:
101102
102103.. code-block :: python
103104
105+ import paddle.batch
104106 import paddle.fluid as fluid
105107 import numpy as np
106108
@@ -109,8 +111,8 @@ PyReader对象通过 :code:`decorate_paddle_reader()` 或 :code:`decorate_tensor
109111 # Case 1: Use decorate_paddle_reader() method to set the data source of py_reader
110112 # The generator yields Numpy-typed batched data
111113 def fake_random_numpy_reader ():
112- image = np.random.random(size = (BATCH_SIZE , 784 ))
113- label = np.random.random_integers(size = (BATCH_SIZE , 1 ), low = 0 , high = 9 )
114+ image = np.random.random(size = (784 , ))
115+ label = np.random.random_integers(size = (1 , ), low = 0 , high = 9 )
114116 yield image, label
115117
116118 py_reader1 = fluid.layers.py_reader(
@@ -120,19 +122,14 @@ PyReader对象通过 :code:`decorate_paddle_reader()` 或 :code:`decorate_tensor
120122 name = ' py_reader1' ,
121123 use_double_buffer = True )
122124
123- py_reader1.decorate_paddle_reader(fake_random_reader)
125+ py_reader1.decorate_paddle_reader(paddle.batch(fake_random_reader, batch_size = BATCH_SIZE ))
126+
124127
125128 # Case 2: Use decorate_tensor_provider() method to set the data source of py_reader
126129 # The generator yields Tensor-typed batched data
127130 def fake_random_tensor_provider ():
128131 image = np.random.random(size = (BATCH_SIZE , 784 )).astype(' float32' )
129132 label = np.random.random_integers(size = (BATCH_SIZE , 1 ), low = 0 , high = 9 ).astype(' int64' )
130-
131- image_tensor = fluid.LoDTensor()
132- image_tensor.set(image, fluid.CPUPlace())
133-
134- label_tensor = fluid.LoDTensor()
135- label_tensor.set(label, fluid.CPUPlace())
136133 yield image_tensor, label_tensor
137134
138135 py_reader2 = fluid.layers.py_reader(
0 commit comments