iron-array

Setup

Clone

git clone --recurse-submodules [email protected]:inaos/iron-array.git

Git commit-hooks

Execute the following commands:

  cp conf/pre-commit .git/hooks/

Build

We use inac cmake build-system in combination with different libraries which can be installed using miniconda3. In particular, one can install LLVM from the numba channel, and MKL and SVML from Intel channel in a cross-platform portable way with:

$ conda install 'llvmdev>=13'
$ conda install -c intel mkl-include
$ conda install -c intel mkl-static
$ conda install -c intel icc_rt    # SVML

Note: it looks like recent versions of conda (when using MacOSX at least) have dependency issues when installing the provious packages. You can find a workaround by using mamba instead. You can install mamba with:

$ conda install mamba -n base -c conda-forge

It is worth noting that conda-forge channel should only be used for installing mamba. In particular, I have detected issues when using the llvmdev package in conda-forge!

Beware: currently ironArray only supports LLVM 11. Also, we strongly suggest to use the numba channel with conda/mamba for installing the LLVM package. In particular, we discourage the use of native LLVM libraries in the system (either using apt, brew or any other packager), even if they are installing LLVM 10 (the numba team seems to be doing a great job in packaging).

Windows

• INAC build setup

  • Make sure that you have a configured repository.txt file in ~.inaos\cmake
  • Also you'll need a directory ~\INAOS (can be empty)

• Create a build folder

     mkdir build
     cd build
  

• Invoke CMAKE, we have to define the generator as well as the build-type

     cmake -G"Visual Studio 14 2015 Win64" -DCMAKE_BUILD_TYPE=Debug ..
     cmake -G"Visual Studio 14 2015 Win64" -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
  

Mac

• INAC build setup:

  • Make sure that you have a configured repository.txt file in ~/.inaos/cmake
  • Also you'll need a directory ~/INAOS (can be empty)

• Create a build folder:

     mkdir build
     cd build
  

• Invoke CMAKE, we have to define the build-type:

     cmake -DCMAKE_BUILD_TYPE=Debug ..
     cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
  

Linux

• INAC build setup

  • Make sure that you have a configured repository.txt file in ~/.inaos/cmake
  • Also you'll need a directory ~/INAOS (can be empty)

• MKL setup. For Ubuntu machines, it is best to use Intel's Ubuntu repo (but you can use conda packages described above too):

     wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB
     apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB
     sudo sh -c 'echo deb https://apt.repos.intel.com/mkl all main > /etc/apt/sources.list.d/intel-mkl.list'
     sudo apt-get update && sudo apt-get install intel-mkl-64bit-2019.X
  

• Create a build folder

     mkdir build
     cd build
  

• Invoke CMAKE, we have to define the build-type, but only two types are supported

     cmake -DCMAKE_BUILD_TYPE=Debug ..
     cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
  

• Some Linux machines (ClearLinux, Gentoo?) require the use of llvm-config utility. You can enforce its use with -DDISABLE_LLVM_CONFIG=False::

     cmake -DCMAKE_BUILD_TYPE=Debug -DDISABLE_LLVM_CONFIG=False ..
     cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DDISABLE_LLVM_CONFIG=False ..
  

Tracing

Sometimes it is useful to activate the tracing mechanism for debugging purposes. Example:

$ env INAC_TRACE='*' ./perf_vector_expression  -e 1 -E 2 -M 3 -t 10 -l 0 -c 9 -f 2
Time for computing and filling X values: 0.0523 s, 2918.5 MB/s
Time for compressing and *storing* X values: 0.106 s, 1434.1 MB/s
Compression for X values: 152.6 MB -> 11.2 MB (13.6x)
Time for computing and filling Y values: 0.0665 s, 2296.2 MB/s
Time for compressing and *storing* Y values: 0.135 s, 1130.3 MB/s
Compression for Y values: 152.6 MB -> 152.6 MB (1.0x)
[iarray.error] - Error compressing a blosc chunk /Users/faltet/inaos/iron-array/src/iarray_expression.c:853
Error during evaluation.  Giving up...

Expressions

• For now only element-wise operations are supported in expressions.

• The iron-array library supports disabling the SVML optimization by setting a DISABLE_SVML environment variable to any value. This can be useful for debugging purposes.