Skip to content

Commit 3adbfd1

Browse files
windstampwindstamp
committed
Add custom device en docs
1 parent 946e985 commit 3adbfd1

17 files changed

+2196
-0
lines changed

‎docs/dev_guides/custom_device_docs/custom_device_example_en.md‎

Lines changed: 462 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/dev_guides/custom_device_docs/custom_kernel_docs/context_api_en.md‎

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# Context APIs
2+
3+
## CustomContext
4+
`CustomContext` is the acutal parameter of the template parameter Context of the custom kernel function. For details, please refer to [custom_context.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/backends/custom/custom_context.h).
5+
6+
```c++
7+
// Constructor
8+
// Parameter：place - CustomPlace object
9+
// Return：None
10+
explicit CustomContext(const CustomPlace&);
11+
12+
// Destructor
13+
virtual ~CustomContext();
14+
15+
// Get the contextual place in the device
16+
// Parameter：None
17+
// Return：place - Place object
18+
const Place& GetPlace() const override;
19+
20+
// Get the contextual stream in the device
21+
// Parameter：None
22+
// Return：stream - void* pointer
23+
void* stream() const;
24+
25+
// Wait for the completion of operations on the stream
26+
// Parameter：None
27+
// Return：None
28+
void Wait() const override;
29+
```
30+
31+
## DeviceContext
32+
`CustomContext` originates from `DeviceContextp`,please refer to [device_context.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/core/device_context.h)
33+
34+
```c++
35+
// No-Parameter constructor
36+
DeviceContext();
37+
38+
// Copy constructor
39+
DeviceContext(const DeviceContext&);
40+
41+
// Move constructor
42+
DeviceContext(DeviceContext&&);
43+
44+
// Move assignment operator
45+
DeviceContext& operator=(DeviceContext&&);
46+
47+
// Destructor
48+
virtual ~DeviceContext();
49+
50+
// Set device allocator
51+
// Parameter：Allocator pointer
52+
// Return：None
53+
void SetAllocator(const Allocator*);
54+
55+
// Set host allocator
56+
// Parameter：Allocator pointer
57+
// Return：None
58+
void SetHostAllocator(const Allocator*);
59+
60+
// Set zero-size allocator
61+
// Parameter：Allocator pointer
62+
// Return：None
63+
void SetZeroAllocator(const Allocator*);
64+
65+
// Get Allocator
66+
// Parameter：None
67+
// Return：Allocator object
68+
const Allocator& GetAllocator() const;
69+
70+
// Get Host allocator
71+
// Parameter：None
72+
// Return：Allocator object
73+
const Allocator& GetHostAllocator() const;
74+
75+
// Get zero-size allocator
76+
// Parameter：None
77+
// Return：Allocator object
78+
const Allocator& GetZeroAllocator() const;
79+
80+
// Allocate the device memory for Tensor
81+
// Parameter: TensorBase pointer
82+
// dtype - DataType variable
83+
// requested_size - size_t variable with the default value of 0
84+
// Return：data pointer - void* pointer
85+
void* Alloc(TensorBase*, DataType dtype, size_t requested_size = 0) const;
86+
87+
// Allocate device memory for Tensor
88+
// Template Parameter：T - data type
89+
// Parameter：TensorBase pointer
90+
// requested_size - size_t variable, 0 by default
91+
// Return：data pointer - T* pointer
92+
template <typename T>
93+
T* Alloc(TensorBase* tensor, size_t requested_size = 0) const;
94+
95+
// Allocate host memory for Tensor
96+
// Parameter：TensorBase pointer
97+
// dtype - DataType variable
98+
// requested_size - size_t variable, 0 by default
99+
// Return：data pointer - void* pointer
100+
void* HostAlloc(TensorBase* tensor,
101+
DataType dtype,
102+
size_t requested_size = 0) const;
103+
104+
// Allocate host storage for Tensor
105+
// Template Parameter：T - data type
106+
// Parameter：TensorBase pointer
107+
// requested_size - size_t variable, 0 by default
108+
// Return：data pointer - T* data pointer
109+
template <typename T>
110+
T* HostAlloc(TensorBase* tensor, size_t requested_size = 0) const;
111+
112+
// Get the contextual information of the place, and implement sub interfaces
113+
// Parameter：None
114+
// Return：place - Place object
115+
virtual const Place& GetPlace() const = 0;
116+
117+
// Wait for the completion of operations on the stream, and implement sub interfaces
118+
// Parameter：None
119+
// Return：None
120+
virtual void Wait() const {}
121+
122+
// Set the random number generator
123+
// Parameter：Generator pointer
124+
// Return：None
125+
void SetGenerator(Generator*);
126+
127+
// Get the random number generator
128+
// Parameter：None
129+
// Return：Generator pointer
130+
Generator* GetGenerator() const;
131+
132+
// Set the Host random number generator
133+
// Parameter：Generator pointer
134+
// Return：None
135+
void SetHostGenerator(Generator*);
136+
137+
// Get the Host random number generator
138+
// Parameter：None
139+
// Return：Generator pointer
140+
Generator* GetHostGenerator() const;
141+
142+
```
143+
144+
## Relevant Information
145+
146+
- `Place` and `CustomPlace`：please refer to [place.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/common/place.h)
147+
- `Allocation` and `Allocator`：please refer to [allocator.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/core/allocator.h)
148+
- `TensorBase`：please refer to [tensor_base.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/core/tensor_base.h)
149+
- `DataType`：please refer to [data_type.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/common/data_type.h)
150+
- `Generator`：please refer to [generator.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/core/generator.h)

‎docs/dev_guides/custom_device_docs/custom_kernel_docs/cpp_api_en.rst‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#############
2+
Kernel Implementation APIs
3+
#############
4+
5+
The custom kernel-function implementation mainly depends on two parts: 1.APIs released by PaddlePaddle, including the context API, the tensor API, and the exception API; 2. APIs of the device encapsulation library. And the C++ API of PaddlePaddle has been released by the header file.
6+
7+
8+
- `Context API <./context_api_en.html>`_ : about the C++ API of the device context
9+
- `Tensor API <./tensor_api_en.html>`_ : about the C++ API of Tensor
10+
- `Exception API <./exception_api_en.html>`_ : about the C++ API of exception handling
11+
12+
13+
Note：There are abundant C++ API of PaddlePaddle. Three APIs will be introduced here and related classes and documents listed in corresponding websites are provided for developers.
14+
15+
.. toctree::
16+
:hidden:
17+
18+
context_api_en.md
19+
tensor_api_en.md
20+
exception_api_en.md

‎docs/dev_guides/custom_device_docs/custom_kernel_docs/exception_api_en.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Exception API
2+
3+
4+
## PADDLE_ENFORCE
5+
6+
How to use：
7+
8+
```c++
9+
PADDLE_ENFORCE_{TYPE}(cond_a, // Condition A
10+
cond_b, // Condition B, optional based on the TYPE
11+
phi::errors::{ERR_TYPE}("{ERR_MSG}"));
12+
```
13+
14+
There are different conditions according to `TYPE`：
15+
16+
| Exception Macro | Basis | Error |
17+
|---|---|---|
18+
| PADDLE_ENFORCE_EQ | cond_a == cond_b | Raise ERR_TYPE exception and report ERR_MSG |
19+
| PADDLE_ENFORCE_NE | cond_a != cond_b | Raise ERR_TYPE exception and report ERR_MSG |
20+
| PADDLE_ENFORCE_GT | cond_a > cond_b | Raise ERR_TYPE exception and report ERR_MSG |
21+
| PADDLE_ENFORCE_GE | cond_a >= cond_b | Raise ERR_TYPE exception and report ERR_MSG |
22+
| PADDLE_ENFORCE_LT | cond_a < cond_b | Raise ERR_TYPE exception and report ERR_MSG |
23+
| PADDLE_ENFORCE_LE | cond_a <= cond_b | Raise ERR_TYPE exception and report ERR_MSG |
24+
| PADDLE_ENFORCE_NOT_NULL | cond_a != nullptr | Raise ERR_TYPE exception and report ERR_MSG |
25+
26+
`ERR_TYPE` supports：
27+
28+
| Type |
29+
|---|
30+
| InvalidArgument |
31+
| NotFound |
32+
| OutOfRange |
33+
| AlreadyExists |
34+
| ResourceExhausted |
35+
| PreconditionNotMet |
36+
| PermissionDenied |
37+
| ExecutionTimeout |
38+
| Unimplemented |
39+
| Unavailable |
40+
| Fatal |
41+
| External |
42+
43+
`ERR_MSG` is a C-style string C, supporting variable-length arguments.
44+
45+
Example：
46+
47+
```c++
48+
// If num_col_dims >= 2 && num_col_dims <= src.size() is not true, report the InvalidArgument exception.
49+
// Print relevant tips
50+
PADDLE_ENFORCE_EQ(
51+
(num_col_dims >= 2 && num_col_dims <= src.size()),
52+
true,
53+
phi::errors::InvalidArgument("The num_col_dims should be inside [2, %d] "
54+
"in flatten_to_3d, but received %d.",
55+
src.size(),
56+
num_col_dims));
57+
```
58+
59+
## Relevant Information
60+
61+
- `PADDLE_ENFORCE`：please refer to [enforce.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/core/enforce.h)
62+
- `errors`：please refer to [errors.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/core/errors.h)

‎docs/dev_guides/custom_device_docs/custom_kernel_docs/kernel_declare_en.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Kernel Function Declaration
2+
3+
PaddlePaddle has released the kernel declaration through the header file, and the framework is uniform both inside and outside.
4+
5+
Custom kernel editing should be based on a specific kernel function declaration. The header file is under `include/paddle/phi/kernels/`.
6+
7+
The format of the declaration is as follows：
8+
9+
```c++
10+
template <typename T, typename Context>
11+
void KernelNameKernel(const Context& dev_ctx,
12+
InputTensor(s),
13+
Attribute(s),
14+
OutTensor(s));
15+
```
16+
17+
Agreement：
18+
19+
1. Template Parameter：It is fixed in format. The data type of the first parameter is `T`，and that of the second is `Context`.
20+
2. Return：`void` is the pattern.
21+
3. Naming：Camel case: kernel name + "Kernel"，such as `SoftmaxKernel`
22+
4. Parameter：Context parameter, InputTensor，Attribute，and OutTensor, all arranged in order:
23+
- Context Parameter：It belongs to `const Context&`.
24+
- `CustomContext` corresponding with the custom kernel. You can refer to [custom_context.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/backends/custom/custom_context.h)
25+
- InputTensor：Number >=0，and the types include：
26+
- `const DenseTensor&` Please refer to [dense_tensor.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/core/dense_tensor.h)
27+
- `const SelectedRows&` Please refer to [selected_rows.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/core/selected_rows.h)
28+
- `const SparseCooTensor&` Please refer to [sparse_coo_tensor.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/core/sparse_coo_tensor.h)
29+
- `const SparseCsrTensor&` Please refer to [sparse_csr_tensor.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/core/sparse_csr_tensor.h)
30+
- `const std::vector<DenseTensor*>&`
31+
- `const std::vector<SparseCooTensor*>&`
32+
- `const std::vector<SparseCsrTensor*>&`
33+
- Attribute：Number >=0，and the types include：
34+
- `bool`
35+
- `float`
36+
- `double`
37+
- `int`
38+
- `int64_t`
39+
- `phi::dtype::float16` Please refer to [float16.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/common/float16.h)
40+
- `const Scalar&` Please refer to [scalar.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/common/scalar.h)
41+
- `DataType` Please refer to [data_type.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/common/data_type.h)
42+
- `DataLayout` Please refer to [layout.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/common/layout.h)
43+
- `Place` Please refer to [place.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/common/place.h)
44+
- `const std::vector<int64_t>&`
45+
- `const ScalarArray&` Please refer to [scalar_array.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/common/scalar_array.h)
46+
- `const std::vector<int>&`
47+
- `const std::string&`
48+
- `const std::vector<bool>&`
49+
- `const std::vector<float>&`
50+
- `const std::vector<double>&`
51+
- `const std::vector<std::string>&`
52+
- OutTensor：Number >0，and the types include：
53+
- `DenseTensor*`
54+
- `SelectedRows*`
55+
- `SparseCooTensor*`
56+
- `SparseCsrTensor*`
57+
- `std::vector<DenseTensor*>`
58+
- `std::vector<SparseCooTensor*>`
59+
- `std::vector<SparseCsrTensor*>`
60+
61+
For example，when the kernel function of `softmax` is in `softmax_kernel.h`:
62+
63+
```c++
64+
// Softmax
65+
// Template Parameter： T - data type
66+
// Context - the device context
67+
// Parameter： dev_ctx - object of the Context
68+
// x - DenseTensor object
69+
// axis - int type
70+
// dtype - DataType type
71+
// out - DenseTensor pointer
72+
// Return： None
73+
template <typename T, typename Context>
74+
void SoftmaxKernel(const Context& dev_ctx,
75+
const DenseTensor& x,
76+
int axis,
77+
DataType dtype,
78+
DenseTensor* out);
79+
```
80+
81+
> Note：
82+
> 1. The kernel function declaration is the basis of the registration and the framework invocation of the custom kernel. It is released by the framework and required to be observed.
83+
> 2. The kernel function declaration cannot perfectly match the header file. You can find the declaration you need by searching the name of the function.

‎docs/dev_guides/custom_device_docs/custom_kernel_docs/register_api_en.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Kernel Registration API
2+
3+
The registration macro of PaddlePaddle helps to register the custom kernel，which can be called by the PaddlePaddle framework.
4+
5+
The registration macro should be put in a global space.
6+
7+
For the basic format of the registration macro, please refer to [kernel_registry.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/core/kernel_registry.h)
8+
9+
```c++
10+
/** PD_REGISTER_PLUGIN_KERNEL
11+
*
12+
* Used to register kernels for plug-in backends.
13+
* Support user-defined backend such as 'Ascend910'.
14+
*/
15+
PD_REGISTER_PLUGIN_KERNEL(kernel_name, backend, layout, meta_kernel_fn, ...)) {}
16+
```
17+
18+
Explanation：
19+
20+
- Name of the macro：`PD_REGISTER_PLUGIN_KERNEL`
21+
- First parameter：kernel_name，which is the same both inside and outside. You can refer to registration names of the same kernel functions of CPU, such as `softmax`.
22+
- Second parameter：backend，which can be customized. But its name must be the same as that of the custom runtime, such as `Ascend910`.
23+
- Third parameter：layout，the enumeration of `DataLayout`. For the setting, please refer to [layout.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/common/layout.h)
24+
- Fourth parameter：meta_kernel_fn，the name of a kernel function. Here, the template parameter is not included, such as `my_namespace::SoftmaxKernel`.
25+
- Variable-length data type parameter: includes basic C++ data types or types defined by PaddlePaddle like `phi::dtype::float16`、`phi::dtype::bfloat16`、`phi::dtype::complex`. You can refer to [data_type.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/common/data_type.h)
26+
- End：the function body. You can set the kernel if necessary. If not, keep `{}`.
27+
28+
>Explanation: the declaration corresponding to the end function body：
29+
>```c++
30+
>// Kernel Parameter Definition
31+
>// Parameter： kernel_key - KernelKey object
32+
>// kernel - Kernel pointer
33+
>// Return： None
34+
>void __PD_KERNEL_args_def_FN_##kernel_name##_##backend##_##layout(
35+
> const ::phi::KernelKey& kernel_key, ::phi::Kernel* kernel);
36+
>```
37+
> You can use the parameters `kernel_key` and `kernel` in the function body，and customize the kernel in its registration.
38+
39+
Take the registration of the CustomCPU backend kernel of `softmax` as an example:
40+
41+
```c++
42+
// The registration of the CustomCPU backend kernel of `softmax`
43+
// Global naming space
44+
// Parameter： softmax - Kernel name
45+
// CustomCPU - Backend name
46+
// ALL_LAYOUT - Storage layout
47+
// custom_cpu::SoftmaxKernel - Kernel function name
48+
// float - name of the data type
49+
// double - name of the data type
50+
// phi::dtype::float16 - name of the data type
51+
PD_REGISTER_PLUGIN_KERNEL(softmax,
52+
CustomCPU,
53+
ALL_LAYOUT,
54+
custom_cpu::SoftmaxKernel,
55+
float,
56+
double,
57+
phi::dtype::float16) {}
58+
```
59+
60+
> Note：
61+
> 1. When the backend is accessed through the custom runtime, the backend parameter must be the same as its name.
62+
> 2. Except the requirement of the end function body of the registration macro，keep the empty function body. You can refer to other backends within the PaddlePaddle framework.

0 commit comments

Comments
 (0)