Merge pull request #1097 from IntelPython/docs/release-0.20-rev01

More revisions and additions to the documentation b3d6cdc

Author: github-actions[bot]

dev/_sources/api_reference/index.rst.txt

@@ -4,3 +4,5 @@
 
 API Reference
 =============
+
+Coming soon

dev/_sources/getting_started.rst.txt

@@ -9,26 +9,40 @@ Getting Started
 ===============
 
 
-Installing pre-built packages
------------------------------
+Installing pre-built conda packages
+-----------------------------------
 
 ``numba-dpex`` along with its dependencies can be installed using ``conda``.
 It is recommended to use conda packages from the ``anaconda.org/intel`` channel
-to get the latest production releases. Nighly builds of ``numba-dpex`` are
-available on the ``dppy/label/dev`` conda channel.
+to get the latest production releases.
 
 .. code-block:: bash
 
-    conda create -n numba-dpex-env numba-dpex dpnp dpctl dpcpp-llvm-spirv spirv-tools -c intel -c conda-forge
+    conda create -n numba-dpex-env                                             \
+        numba-dpex dpnp dpctl dpcpp-llvm-spirv spirv-tools                     \
+        -c intel -c conda-forge
+
+To try out the bleeding edge, the latest packages built from tip of the main
+source trunk can be installed from the ``dppy/label/dev`` conda channel.
+
+.. code-block:: bash
+
+    conda create -n numba-dpex-env                                             \
+        numba-dpex dpnp dpctl dpcpp-llvm-spirv spirv-tools                     \
+        -c dppy/label/dev -c intel -c conda-forge
+
+
 
 Building from source
 --------------------
 
-``numba-dpex`` can be built from source using either ``conda-build`` or ``setuptools``.
+``numba-dpex`` can be built from source using either ``conda-build`` or
+``setuptools``.
 
 Steps to build using ``conda-build``:
 
-1. Create a conda environment
+1. Ensure ``conda-build`` is installed in the ``base`` conda environment or
+   create a new conda environment with ``conda-build`` installed.
 
 .. code-block:: bash
 
@@ -45,22 +59,34 @@ Steps to build using ``conda-build``:
 
 .. code-block:: bash
 
-    conda install numba-dpex
+    conda install -c local numba-dpex
 
 Steps to build using ``setup.py``:
 
+As before, a conda environment with all necessary dependencies is the suggested
+first step.
+
 .. code-block:: bash
 
-    conda create -n numba-dpex-env dpctl dpnp numba spirv-tools dpcpp-llvm-spirv llvmdev pytest -c intel -c conda-forge
+    # Create a conda environment that hass needed dependencies installed
+    conda create -n numba-dpex-env                                             \
+        dpctl dpnp numba spirv-tools dpcpp-llvm-spirv llvmdev pytest           \
+        -c intel -c conda-forge
+    # Activate the environment
     conda activate numba-dpex-env
+    # Clone the numba-dpex repository
+    git clone https://github.com/IntelPython/numba-dpex.git
+    cd numba-dpex
+    python setup.py develop
 
 Building inside Docker
 ----------------------
 
-A Dockerfile is provided on the GitHub repository to easily build ``numba-dpex``
+A Dockerfile is provided on the GitHub repository to build ``numba-dpex``
 as well as its direct dependencies: ``dpctl`` and ``dpnp``. Users can either use
 one of the pre-built images on the ``numba-dpex`` GitHub page or use the
-bundled Dockerfile to build ``numba-dpex`` from source.
+bundled Dockerfile to build ``numba-dpex`` from source. Using the Dockerfile
+also ensures that all device drivers and runtime libraries are pre-installed.
 
 Building
 ~~~~~~~~
@@ -69,10 +95,10 @@ Numba dpex ships with multistage Dockerfile, which means there are
 different `targets <https://docs.docker.com/build/building/multi-stage/#stop-at-a-specific-build-stage>`_
 available for build. The most useful ones:
 
-- runtime
-- runtime-gpu
-- numba-dpex-builder-runtime
-- numba-dpex-builder-runtime-gpu
+- ``runtime``
+- ``runtime-gpu``
+- ``numba-dpex-builder-runtime``
+- ``numba-dpex-builder-runtime-gpu``
 
 To build docker image
 
@@ -96,7 +122,7 @@ To run docker image
     ``GITHUB_USER`` and ``GITHUB_PASSWORD``
     `build args <https://docs.docker.com/engine/reference/commandline/build/#build-arg>`_
     to increase the call limit. A GitHub
-    `access token <https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token>`
+    `access token <https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token>`_
     can also be used instead of the password.
 
 .. note::

dev/_sources/overview.rst.txt

@@ -15,23 +15,23 @@ implementation of `NumPy*`_'s API using the `SYCL*`_ language.
 .. the same time automatically running such code parallelly on various types of
 .. architecture.
 
-``numba-dpex`` is developed as part of `Intel AI Analytics Toolkit`_ and
-is distributed with the `Intel Distribution for Python*`_. The extension is
-available on Anaconda cloud and as a Docker image on GitHub. Please refer the
-:doc:`getting_started` page to learn more.
+``numba-dpex`` is an open-source project and can be installed as part of `Intel
+AI Analytics Toolkit`_ or the `Intel Distribution for Python*`_. The package is
+also available on Anaconda cloud and as a Docker image on GitHub. Please refer
+the :doc:`getting_started` page to learn more.
 
 Main Features
 -------------
 
 Portable Kernel Programming
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The ``numba-dpex`` kernel API has a design and API similar to Numba's
+The ``numba-dpex`` kernel programming API has a design similar to Numba's
 ``cuda.jit`` sub-module. The API is modeled after the `SYCL*`_ language and uses
 the `DPC++`_ SYCL runtime. Currently, compilation of kernels is supported for
 SPIR-V-based OpenCL and `oneAPI Level Zero`_ devices CPU and GPU devices. In the
-future, the API can be extended to other architectures that are supported by
-DPC++.
+future, compilation support for other types of hardware that are supported by
+DPC++ will be added.
 
 The following example illustrates a vector addition kernel written with
 ``numba-dpex`` kernel API.
@@ -56,31 +56,33 @@ The following example illustrates a vector addition kernel written with
     print(c)
 
 In the above example, three arrays are allocated on a default ``gpu`` device
-using the ``dpnp`` library. These arrays are then passed as input arguments to
-the kernel function. The compilation target and the subsequent execution of the
-kernel is determined completely by the input arguments and follow the
+using the ``dpnp`` library. The arrays are then passed as input arguments to the
+kernel function. The compilation target and the subsequent execution of the
+kernel is determined by the input arguments and follow the
 "compute-follows-data" programming model as specified in the `Python* Array API
 Standard`_. To change the execution target to a CPU, the device keyword needs to
 be changed to ``cpu`` when allocating the ``dpnp`` arrays. It is also possible
 to leave the ``device`` keyword undefined and let the ``dpnp`` library select a
 default device based on environment flag settings. Refer the
 :doc:`user_guide/kernel_programming/index` for further details.
 
-``dpnp`` compilation support
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-``numba-dpex`` extends Numba's type system and compilation pipeline to compile
-``dpnp`` functions and expressions in the same way as NumPy. Unlike Numba's
-NumPy compilation that is serial by default, ``numba-dpex`` always compiles
-``dpnp`` expressions into data-parallel kernels and executes them in parallel.
-The ``dpnp`` compilation feature is provided using a decorator ``dpjit`` that
-behaves identically to ``numba.njit(parallel=True)`` with the addition of
-``dpnp`` compilation and kernel offloading. Offloading by ``numba-dpex`` is not
-just restricted to CPUs and supports all devices that are presently supported by
-the kernel API. ``dpjit`` allows using NumPy and ``dpnp`` expressions in the
-same function. All NumPy compilation and parallelization is done via the default
-Numba code-generation pipeline, whereas ``dpnp`` expressions are compiled using
-the ``numba-dpex`` pipeline.
+``dpjit`` decorator
+~~~~~~~~~~~~~~~~~~~
+
+The ``numba-dpex`` package provides a new decorator ``dpjit`` that extends
+Numba's ``njit`` decorator. The new decorator is equivalent to
+``numba.njit(parallel=True)``, but additionally supports compiling ``dpnp``
+functions, ``prange`` loops, and array expressions that use ``dpnp.ndarray``
+objects.
+
+Unlike Numba's NumPy parallelization that only supports CPUs, ``dpnp``
+expressions are first converted to data-parallel kernels and can then be
+`offloaded` to different types of devices. As ``dpnp`` implements the same API
+as NumPy*, an existing ``numba.njit`` decorated function that uses
+``numpy.ndarray`` may be refactored to use ``dpnp.ndarray`` and decorated with
+``dpjit``. Such a refactoring can allow the parallel regions to be offloaded
+to a supported GPU device, providing users an additional option to execute their
+code parallelly.
 
 The vector addition example depicted using the kernel API can also be
 expressed in several different ways using ``dpjit``.

dev/_sources/user_guide/caching.rst.txt

@@ -1,48 +1,49 @@
-.. _caching:
 .. include:: ./../ext_links.txt
 
-
-Caching Mechanism in Numba-dpex
-================================
+Caching Mechanism in ``numba-dpex``
+===================================
 
 Caching is done by saving the compiled kernel code, the ELF object of the
 executable code. By using the kernel code, cached kernels have minimal overhead
 because no compilation is needed.
 
-Unlike Numba, we do not perform file-based caching, instead we use a
-Least Recently Used (LRU) caching mechanism. However when a kernel needs to be
-evicted, we utilize numba's file-based caching mechanism described
-`here <https://numba.pydata.org/numba-doc/latest/developer/caching.html>`_.
+Unlike Numba*, ``numba-dpex`` does not perform an exclusive file-based caching,
+instead a Least Recently Used (LRU) caching mechanism is used. However, when a
+kernel needs to be evicted, Numba's file-based caching mechanism is invoked as
+described `here
+<https://numba.pydata.org/numba-doc/latest/developer/caching.html>`_.
 
 Algorithm
----------
-
-The caching mechanism for Numba-dpex works as follows: The cache is an LRU cache
-backed by an ordered dictionary mapped onto a doubly linked list. The tail of
-the list contains the most recently used (MRU) kernel and the head of the list
-contains the least recently used (LRU) kernel. The list  has a fixed size. If a
-new kernel arrives to be cached and if the size is already on the maximum limit,
-the algorithm evicts the LRU kernel to make room for the MRU kernel. The evicted
-item will be serialized and pickled into a file using Numba's caching mechanism.
-
-Everytime whenever a kernel needs to be retrieved from the cache, the mechanism
-will look for the kernel in the cache and will be loaded if it's already present.
-However, if the program is seeking for a kernel that has been evicted, the
-algorithm will load it from the file and enqueue in the cache.
+----------
+
+The caching mechanism for ``numba-dpex`` works as follows: The cache is an LRU
+cache backed by an ordered dictionary mapped onto a doubly linked list. The tail
+of the list contains the most recently used (MRU) kernel and the head of the
+list contains the least recently used (LRU) kernel. The list  has a fixed size.
+If a new kernel arrives to be cached and if the size is already on the maximum
+limit, the algorithm evicts the LRU kernel to make room for the MRU kernel. The
+evicted item will be serialized and pickled into a file using Numba's caching
+mechanism.
+
+Everytime when a kernel needs to be retrieved from the cache, the mechanism
+will look for the kernel in the cache and will be loaded if it's already
+present. However, if the program is seeking for a kernel that has been evicted,
+the algorithm will load it from the file and enqueue in the cache. As a result,
+the amount of file operations are significantly lower than that of Numba.
 
 Settings
---------
+---------
 
-Therefore, we employ similar environment variables as used in Numba,
-i.e. ``NUMBA_CACHE_DIR`` etc. However we add three more environment variables to
-control the caching mechanism.
+Therefore, ``numba-dpex`` employs similar environment variables as used in
+Numba, i.e. ``NUMBA_CACHE_DIR`` etc. However there are three more environment
+variables to control the caching mechanism.
 
-- In order to specify cache capacity, one can use ``NUMBA_DPEX_CACHE_SIZE``.
-By default, it's set to 10.
+- In order to specify cache capacity, ``NUMBA_DPEX_CACHE_SIZE`` can be used. By
+  default, it's set to 10.
 
-- ``NUMBA_DPEX_ENABLE_CACHE`` can be used to enable/disable the caching mechanism.
-By default it's enabled, i.e. set to 1.
+- ``NUMBA_DPEX_ENABLE_CACHE`` can be used to enable/disable the caching
+  mechanism. By default it's enabled, i.e. set to 1.
 
-- In order to enable the debugging messages related to caching, one can set
-``NUMBA_DPEX_DEBUG_CACHE`` to 1. All environment variables are defined in
-:file:`numba_dpex/config.py`.
+- In order to enable the debugging messages related to caching, the variable
+``NUMBA_DPEX_DEBUG_CACHE`` can be set to 1. All environment variables are
+defined in :file:`numba_dpex/config.py`.

dev/_sources/user_guide/config.rst.txt

@@ -0,0 +1,62 @@
+.. include:: ./../ext_links.txt
+
+Configuration Options for ``numba-dpex``
+========================================
+
+``numba-dpex`` provides a set of environment variables and flags for configuring different aspects of the compilation, debugging and execution of programs. The configuration flags of ``numba-dpex`` are mostly inherited from those of Numba*. They are defined in :file:`numba_dpex/core/config.py`.
+
+.. note::
+    In order to enable/disable each of the configuration flags, a ``NUMBA_DPEX``
+    prefix needs to be appended before each variable. For example, in order to
+    turn ``SAVE_IR_FILES`` flag on, it needs to be passed as ``NUMBA_DPEX_SAVE_IR_FILES=1``
+
+    For example:
+
+    .. code-block:: bash
+
+        user@host:~/NUMBA_DPEX_SAVE_IR_FILES=1 python numba_dpex_program.py
+
+
+The list of available configuration flags are listed as follows:
+
+``SAVE_IR_FILES``:
+    A flag to save the Numba* intermediate representation (IR) files generated by the compiler. Set to ``0`` by default.
+
+``SPIRV_VAL``:
+    A flag to turn Numba*'s ``SPIRV-VALIDATION`` switch. Set to ``0`` by default.
+
+``OFFLOAD_DIAGNOSTICS``:
+    A flag to dump the offload diagnostics. Set to ``0`` by default.
+
+``NATIVE_FP_ATOMICS``:
+    A flag to activate the native floating point (FP) atomcis support for supported devices. Requires ``llvm-spirv`` supporting the FP atomics extension. Set to ``0`` by default.
+
+``DEBUG``:
+    A flag to emit the debug info, inherited from |numba.core.config.DEBUG|_.
+
+``DEBUGINFO_DEFAULT``:
+    The default value for the `debug` flag. Inherited from |numba.core.config.DEBUGINFO_DEFAULT|_.
+
+``DUMP_KERNEL_LLVM``:
+    A flag to emit LLVM assembly language format (``.ll``). Inherited from |numba.core.config.DUMP_OPTIMIZED|_.
+
+``ENABLE_CACHE``:
+    A flag to enable caching, set ``NUMBA_DPEX_ENABLE_CACHE=0`` to turn off. Set to ``1`` by default.
+
+``CACHE_SIZE``:
+    A flag to specify the default cache size. Set to ``20`` by default.
+
+``DEBUG_CACHE``:
+    A flag to enable debugging of cahcing mechanism, set ``1`` to turn it on.
+
+``STATIC_LOCAL_MEM_PASS``:
+    A flag to turn on the ``ConstantSizeStaticLocalMemoryPass`` in the kernel pipeline. The pass is turned off by default.
+
+
+.. |numba.core.config.DEBUG| replace:: ``numba.core.config.DEBUG``
+.. |numba.core.config.DEBUGINFO_DEFAULT| replace:: ``numba.core.config.DEBUGINFO_DEFAULT``
+.. |numba.core.config.DUMP_OPTIMIZED| replace:: ``numba.core.config.DUMP_OPTIMIZED``
+
+.. _`numba.core.config.DEBUG`: https://github.com/numba/numba/blob/main/numba/core/config.py#L202
+.. _`numba.core.config.DEBUGINFO_DEFAULT`: https://github.com/numba/numba/blob/main/numba/core/config.py#L488
+.. _`numba.core.config.DUMP_OPTIMIZED`: https://github.com/numba/numba/blob/main/numba/core/config.py#L301

dev/_sources/user_guide/debugging/altering.rst.txt

@@ -1,4 +1,4 @@
-.. _altering:
+.. include:: ./../../ext_links.txt
 
 Altering Execution
 ==================

dev/_sources/user_guide/debugging/backtrace.rst.txt

@@ -1,4 +1,4 @@
-.. _backtrace:
+.. include:: ./../../ext_links.txt
 
 Backtrace
 ==========

dev/_sources/user_guide/debugging/breakpoints.rst.txt

@@ -1,3 +1,5 @@
+.. include:: ./../../ext_links.txt
+
 Breakpoints
 ===========
 

dev/_sources/user_guide/debugging/common_issues.rst.txt

@@ -1,3 +1,5 @@
+.. include:: ./../../ext_links.txt
+
 Common issues and tips
 ======================
 

dev/_sources/user_guide/debugging/data.rst.txt

@@ -1,3 +1,5 @@
+.. include:: ./../../ext_links.txt
+
 Examining Data
 ==============