intelpython / mkl-service Goto Github PK

Classes could be added that allocate memory using mkl_malloc, with specified alignment, and expose it to Python via buffer protocol.

In _mklinitmodule.c the environment variable MKL_THREADING_LAYER is set. A library should never set any env vars.

This is resulting in issues in conda envs when libgomp is already loaded, see pytorch/pytorch#37377.

Presumably, this is not supported yet here? Would be sure nice to have it as it's available in 2019.3 release.

All the links that I've come across, from the blog post mentioned in the README to links in the code to documentation redirect to either the top-level MKL landing page or a generic Intel developer resources page without any additional context. Is it possible to fix these links so they will work again?

At the moment using pip we are limited to Python 3.9 and with conda to Python 3.9 with intel/mkl-service and 3.10 with anaconda/mkl-service.

At least on Windows and Linux I could successfully build mkl-service manually with Python 3.10 and on Windows also with 3.11.
So this shouldn’t be much effort.

In mkl-service/mkl/init.py, at line 49, _mlkinit is missing, leading to UserWarning while using multiprocessing and joblib libraries.
error message

UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
  from . import _distributor_init

pkg_resources from setuptools is a good, generally robust way to find the version of a package. However, mkl/mkl-service fails for that.

See: pypa/setuptools#1792

According to one of the 'Python Packaging Authority'-guys this is due to missing metadata in mkl:

Sorry, I don't use conda, you'll have to check if the conda package does include the necessary metadata. From manually listing the contents of one of the packages here, that does not seem to be the case.
(pypa/setuptools#1792 (comment))

Here the error message (mkl.__version__ behaves as expected):

In [18]: import mkl                                                             

In [19]: mkl.__version__                                                        
Out[19]: '1.1.2'

In [20]: import pkg_resources                                                   

In [21]: pkg_resources.get_distribution("mkl").version                          
---------------------------------------------------------------------------
DistributionNotFound                      Traceback (most recent call last)
<ipython-input-21-0cf8bac97847> in <module>
----> 1 pkg_resources.get_distribution("mkl").version

~/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py in get_distribution(dist)
    479         dist = Requirement.parse(dist)
    480     if isinstance(dist, Requirement):
--> 481         dist = get_provider(dist)
    482     if not isinstance(dist, Distribution):
    483         raise TypeError("Expected string, Requirement, or Distribution", dist)

~/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py in get_provider(moduleOrReq)
    355     """Return an IResourceProvider for the named module or requirement"""
    356     if isinstance(moduleOrReq, Requirement):
--> 357         return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
    358     try:
    359         module = sys.modules[moduleOrReq]

~/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py in require(self, *requirements)
    898         included, even if they were already activated in this working set.
    899         """
--> 900         needed = self.resolve(parse_requirements(requirements))
    901 
    902         for dist in needed:

~/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py in resolve(self, requirements, env, installer, replace_conflicting, extras)
    784                     if dist is None:
    785                         requirers = required_by.get(req, None)
--> 786                         raise DistributionNotFound(req, requirers)
    787                 to_activate.append(dist)
    788             if dist not in req:

DistributionNotFound: The 'mkl' distribution was not found and is required by the application

Here my system info

In [22]: import scooby                                                          

In [23]: scooby.investigate()                                                   
Out[23]: 

------------------------------------------------------
  Date: Wed Jun 26 13:58:53 2019 CEST
  Platform: Linux-4.18.0-24-generic-x86_64-with-debian-buster-sid

              4 : CPU(s)
         x86_64 : Machine
          64bit : Architecture
        15.6 GB : RAM

  3.7.3 (default, Mar 27 2019, 22:11:17)  [GCC
  7.3.0]

        IPython : Environment

         1.16.2 : numpy
          1.2.1 : scipy
          7.4.0 : IPython
          3.0.3 : matplotlib

  Intel(R) Math Kernel Library Version 2019.0.3
  Product Build 20190125 for Intel(R) 64
  architecture applications
------------------------------------------------------

Is this expected?

>>> mkl.cbwr_set('avx2')
'success'

>>> mkl.cbwr_get()
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
ValueError: Variable can not be None
Exception ignored in: 'mkl._py_mkl_service.__mkl_str_to_int'
ValueError: Variable can not be None
'err_invalid_input'

Hi. Could you please suggest how to fix the error: ‘MKL_ENABLE_AVX2_E1’ undeclared while building mkl-service ?
I'm compiling mkl-services-2.4.0.post1 against mkl-2021.1.1 and openmpp-19.1.3, python 3.8

I'm trying to install pypardiso on Python 3.9, which depends on mkl-service, but pip complains that:

$ pip install pypardiso==0.3.3
Collecting pypardiso==0.3.3
  Using cached pypardiso-0.3.3-py3-none-any.whl (10 kB)
ERROR: Could not find a version that satisfies the requirement mkl-service (from pypardiso) (from versions: none)
ERROR: No matching distribution found for mkl-service

Hello there,

I have tried to get started in Python for quite some time. But I am starting to get frustrated, because it seems like I just can not get it to work.

The Error that caused my latest frustration will be discribed below, but if someone could tell me if that could be an indicator for an underlying problem that I need to solve. Also, if there is some etiquette I violated with this Issue, please let me know.

The following code should be fairly simple

import CoolProp
from CoolProp.CoolProp import PropsSI

PropsSI('T', 'P', 101325, 'Q', 0, 'Water')

The following issue turns up
PS C:\Users\paul_lemke\Documents\02_Python_Programme> & C:/01_Heruntergeladene_Programme/01_Anaconda/envs/py36/python.exe
c:/Users/paul_lemke/Documents/02_Python_Programme/.vscode/Fluid_Properties/Fluid_Property_simple.py
C:\01_Heruntergeladene_Programme\01_Anaconda\envs\py36\lib\site-packages\numpy_init_.py:138: UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
from . import _distributor_init

I dont know what I am doing wrong and I will really appreciate every kind of help.

With best regards

Currently, any exception raised will be ignored and it will print "exception is ignored" to stderr. Then, the user has to manage all those error codes manually (which requires reading Cython source code which may be a bit too much for an average user).

Maybe throw proper Python exceptions instead? That would arguably be a much more Pythonic way of doing that:

https://cython.readthedocs.io/en/latest/src/userguide/language_basics.html#error-return-values

So instead of stuff like

return_code = mkl.set_foo(...)
if return_code != 'success':
    if return_code == 'err_invalid_input':
        raise RuntimeError('invalid input in mkl')

you could do just

mkl.set_foo(...)

which would throw an MKLError('invalid input') if it fails, so you could catch it later.

Here's how to reproduce the bug:

pip install mkl mkl-service
python -c "import mkl"

and here's the output:

Collecting mkl
  Downloading mkl-2021.4.0-py2.py3-none-win_amd64.whl (228.5 MB)
Collecting mkl-service
  Downloading mkl_service-2.4.0-0-cp37-cp37m-win_amd64.whl (43 kB)
Collecting tbb==2021.*
  Downloading tbb-2021.4.0-py3-none-win_amd64.whl (268 kB)
Collecting intel-openmp==2021.*
  Downloading intel_openmp-2021.4.0-py2.py3-none-win_amd64.whl (3.5 MB)
Collecting six
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: tbb, intel-openmp, six, mkl, mkl-service
Successfully installed intel-openmp-2021.4.0 mkl-2021.4.0 mkl-service-2.4.0 six-1.16.0 tbb-2021.4.0
WARNING: You are using pip version 21.3; however, version 21.3.1 is available.
You should consider upgrading via the 'c:\hostedtoolcache\windows\python\3.7.9\x64\python.exe -m pip install --upgrade pip' command.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\mkl\__init__.py", line 49, in <module>
    from . import _mklinit
ImportError: DLL load failed: The specified module could not be found.
Error: Process completed with exit code 1.

Currently:

In [1]: import mkl

In [2]: mkl.vml_get_mode()
Out[2]: ('ha', 'default', 'default')

In [3]: mkl.vml_set_mode(*mkl.vml_get_mode())
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
ValueError: Variable: <default> not in {'on': 2621440, 'off': 1310720}
Exception ignored in: 'mkl._py_mkl_service.__mkl_str_to_int'
ValueError: Variable: <default> not in {'on': 2621440, 'off': 1310720}
Out[3]: ('ha', 'default', 'default')

Due to this few days I cannot use conda to install any packages (use mirrors but no effects), so could we install mkl-service in pip?

Per https://software.intel.com/en-us/node/522114 setting 0 for a number of threads is interpreted as setting it to "the default number of threads for the OpenMP run-time library."

Yet:

$ python -c "import mkl; mkl.domain_set_num_threads(0, domain='fft')"
-c:1: UserWarning: Non-positive argument of domain_set_num_threads is being ignored, number of threads will not be changed

This is a bit of an odd issue. I realized that the new mkl version breaks opencv-python. This issue was previously opened in the opencv-python repo (opencv/opencv-python#308 (comment)), but I was suggested to opened it elsewhere. Hope you can provide some help or point me in the right direction.

Expected behaviour

OpenCV can load openh264-1.8.0-win64.dll

Actual bejaviour

OpenCV cannot load openh264-1.8.0-win64.dll

Steps to reproduce

Create a new conda environment with the following

name: test_opencv

channels:
 - defaults

dependencies:
- python
- mkl==2020.0
- numpy
- pip
- pip:
   - opencv-contrib-python

and run the following code:

import cv2

fourcc = cv2.VideoWriter_fourcc(*'X264')
out = cv2.VideoWriter('output.avi',fourcc, 20.0, (640,480))

By conda installing mkl==2020.0 opencv cannot load openh264-1.8.0-win64.dll. The following error appears:

...
Failed to load OpenH264 library: openh264-1.8.0-win64.dll
        Please check environment and/or download library: https://github.com/cisco/openh264/releases
...

Instead, installing mkl==2019.* via conda the openH264 is loaded successfully:

...
OpenH264 Video Codec provided by Cisco Systems, Inc.
...

• operating system: Windows 10
• architecture (e.g. x86): amd64