Remove strict requirement of MKL so that M1 chips are supported natively

[elbamos]

If you have questions about a specific use case, or you are not sure whether this is a bug or not, please post it to our discourse channel: https://discourse.pymc.io

Description of your problem

% mamba install pymc3=3.11
Encountered problems while solving:
  - nothing provides mkl-service needed by pymc3-3.11.0-pyhd8ed1ab_0

Note that earlier versions of pymc3 fail to install because they depend on theano or theano-pymc, which aren't available in M1 builds, rather than aesara.

It appears from here: 2edbe6c that mkl-service was added to quash a theano warning. Is mkl-service still really necessary with aesara?

[OriolAbril]

pymc3 3.11 still uses theano-pymc, not sure if it helps, but Aesara will be used from 4.0 onwards only.

[twiecki]

@OriolAbril I don't think that will help, aesara still is much faster with mkl.

@elbamos Isn't there a compatibility mode? @fonnesbeck talks about it working well.

[fonnesbeck]

Currently, you won't be able to run this natively. It will have to be under Rosetta2.

[elbamos]

@twiecki It can run under Rosetta2, although if you're also running native python, its a real chore. Pymc is the last tool in my toolbox that still needs to run under compatibility mode. I had hoped this was a conda dependencies issue that could be resolved to permit native operation, but I suppose the answer is that it is not.

[elbamos]

Actually, I take that back.

I was able to get the current version of pymc3 (installed from git, reporting v3.11.1) running natively on M1, simply by changing two references to aesana.tensor.nnet.sigmoid to point to aesana.tensor.sigmoid.

I'm guessing the naming issue is really that what's currently up on the git references a different version of aesana than what's found on conda-forge.

Am I missing something? It really seems that mkl-service doesn't need to be a requirement, and if its dropped, then this can run natively as is.

If someone proposes a benchmark to show the performance implication, I'm happy to run it.

[elbamos]

If anyone wants to try it (@fonnesbeck ?), I've pushed to branch m1 in a fork, and am debating whether to make a PR: https://github.com/elbamos/pymc3/tree/m1

This works with the version of aesana available on conda-forge.

[twiecki]

Revisiting this issue. It seems like indeed we should remove the strict MKL requirement now that M1 exists. Ideally we'd find a way to tell conda to use MKL if it's available but fall back to OpenBLAS if it doesn't.

[twiecki]

@elbamos Want to do a PR?

[twiecki]

https://conda-forge.org/blog/posts/2020-10-29-macos-arm64/#how-to-add-a-osx-arm64-build-to-a-feedstock

[twiecki]

Digging a bit deeper:
I think we should drop the mkl-services dependency. PyMC3 itself does not depend on libblas (like MKL), only aesara does, so it should be on aesara to specify that dependency.

Moreover, we prohibit the use of other libblas implementations which is causing this problem here.

The reason we did is that by default conda install pymc3 pulls in openblas which is much slower than mkl. Instead, however, we should change our installation instructions to say conda install pymc3 libblas=*=*mkl to get the right backend if you need it, but do not prohibit installation on M1.

[elbamos]

@twiecki I think you’re exactly right. My recollection of when I tried to do this to support a pr (which is now way out of date behind master) is that the only required change was dropping mkl-service from the requirements and an install script.