intelpython / mkl-service Goto Github PK
View Code? Open in Web Editor NEWPython hooks for Intel(R) Math Kernel Library runtime control settings.
License: BSD 3-Clause "New" or "Revised" License
Python hooks for Intel(R) Math Kernel Library runtime control settings.
License: BSD 3-Clause "New" or "Revised" License
Hi. Could you please suggest how to fix the error: ‘MKL_ENABLE_AVX2_E1’ undeclared
while building mkl-service ?
I'm compiling mkl-services-2.4.0.post1
against mkl-2021.1.1
and openmpp-19.1.3
, python 3.8
ImportError Traceback (most recent call last)
Cell In[3], line 1
----> 1 import pandas as pd
File ~\New folder\Lib\site-packages\pandas_init_.py:16
13 _missing_dependencies.append(f"{_dependency}: {_e}")
15 if _missing_dependencies: # pragma: no cover
---> 16 raise ImportError(
17 "Unable to import required dependencies:\n" + "\n".join(_missing_dependencies)
18 )
19 del _hard_dependencies, _dependency, _missing_dependencies
21 try:
22 # numpy compat
ImportError: Unable to import required dependencies:
numpy: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.
In _mklinitmodule.c
the environment variable MKL_THREADING_LAYER
is set. A library should never set any env vars.
This is resulting in issues in conda envs when libgomp
is already loaded, see pytorch/pytorch#37377.
Currently, any exception raised will be ignored and it will print "exception is ignored" to stderr. Then, the user has to manage all those error codes manually (which requires reading Cython source code which may be a bit too much for an average user).
Maybe throw proper Python exceptions instead? That would arguably be a much more Pythonic way of doing that:
https://cython.readthedocs.io/en/latest/src/userguide/language_basics.html#error-return-values
So instead of stuff like
return_code = mkl.set_foo(...)
if return_code != 'success':
if return_code == 'err_invalid_input':
raise RuntimeError('invalid input in mkl')
you could do just
mkl.set_foo(...)
which would throw an MKLError('invalid input')
if it fails, so you could catch it later.
All the links that I've come across, from the blog post mentioned in the README to links in the code to documentation redirect to either the top-level MKL landing page or a generic Intel developer resources page without any additional context. Is it possible to fix these links so they will work again?
Presumably, this is not supported yet here? Would be sure nice to have it as it's available in 2019.3 release.
Classes could be added that allocate memory using mkl_malloc
, with specified alignment, and expose it to Python via buffer protocol.
pkg_resources
from setuptools
is a good, generally robust way to find the version of a package. However, mkl
/mkl-service
fails for that.
See: pypa/setuptools#1792
According to one of the 'Python Packaging Authority'-guys this is due to missing metadata in mkl
:
Sorry, I don't use conda, you'll have to check if the conda package does include the necessary metadata. From manually listing the contents of one of the packages here, that does not seem to be the case.
(pypa/setuptools#1792 (comment))
Here the error message (mkl.__version__
behaves as expected):
In [18]: import mkl
In [19]: mkl.__version__
Out[19]: '1.1.2'
In [20]: import pkg_resources
In [21]: pkg_resources.get_distribution("mkl").version
---------------------------------------------------------------------------
DistributionNotFound Traceback (most recent call last)
<ipython-input-21-0cf8bac97847> in <module>
----> 1 pkg_resources.get_distribution("mkl").version
~/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py in get_distribution(dist)
479 dist = Requirement.parse(dist)
480 if isinstance(dist, Requirement):
--> 481 dist = get_provider(dist)
482 if not isinstance(dist, Distribution):
483 raise TypeError("Expected string, Requirement, or Distribution", dist)
~/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py in get_provider(moduleOrReq)
355 """Return an IResourceProvider for the named module or requirement"""
356 if isinstance(moduleOrReq, Requirement):
--> 357 return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
358 try:
359 module = sys.modules[moduleOrReq]
~/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py in require(self, *requirements)
898 included, even if they were already activated in this working set.
899 """
--> 900 needed = self.resolve(parse_requirements(requirements))
901
902 for dist in needed:
~/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py in resolve(self, requirements, env, installer, replace_conflicting, extras)
784 if dist is None:
785 requirers = required_by.get(req, None)
--> 786 raise DistributionNotFound(req, requirers)
787 to_activate.append(dist)
788 if dist not in req:
DistributionNotFound: The 'mkl' distribution was not found and is required by the application
Here my system info
In [22]: import scooby
In [23]: scooby.investigate()
Out[23]:
------------------------------------------------------
Date: Wed Jun 26 13:58:53 2019 CEST
Platform: Linux-4.18.0-24-generic-x86_64-with-debian-buster-sid
4 : CPU(s)
x86_64 : Machine
64bit : Architecture
15.6 GB : RAM
3.7.3 (default, Mar 27 2019, 22:11:17) [GCC
7.3.0]
IPython : Environment
1.16.2 : numpy
1.2.1 : scipy
7.4.0 : IPython
3.0.3 : matplotlib
Intel(R) Math Kernel Library Version 2019.0.3
Product Build 20190125 for Intel(R) 64
architecture applications
------------------------------------------------------
Is this expected?
>>> mkl.cbwr_set('avx2')
'success'
>>> mkl.cbwr_get()
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
ValueError: Variable can not be None
Exception ignored in: 'mkl._py_mkl_service.__mkl_str_to_int'
ValueError: Variable can not be None
'err_invalid_input'
In mkl-service/mkl/init.py, at line 49, _mlkinit is missing, leading to UserWarning while using multiprocessing and joblib libraries.
error message
UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
from . import _distributor_init
At the moment using pip we are limited to Python 3.9 and with conda to Python 3.9 with intel/mkl-service and 3.10 with anaconda/mkl-service.
At least on Windows and Linux I could successfully build mkl-service manually with Python 3.10 and on Windows also with 3.11.
So this shouldn’t be much effort.
Hello there,
I have tried to get started in Python for quite some time. But I am starting to get frustrated, because it seems like I just can not get it to work.
The Error that caused my latest frustration will be discribed below, but if someone could tell me if that could be an indicator for an underlying problem that I need to solve. Also, if there is some etiquette I violated with this Issue, please let me know.
The following code should be fairly simple
import CoolProp
from CoolProp.CoolProp import PropsSI
PropsSI('T', 'P', 101325, 'Q', 0, 'Water')
The following issue turns up
PS C:\Users\paul_lemke\Documents\02_Python_Programme> & C:/01_Heruntergeladene_Programme/01_Anaconda/envs/py36/python.exe
c:/Users/paul_lemke/Documents/02_Python_Programme/.vscode/Fluid_Properties/Fluid_Property_simple.py
C:\01_Heruntergeladene_Programme\01_Anaconda\envs\py36\lib\site-packages\numpy_init_.py:138: UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
from . import _distributor_init
I dont know what I am doing wrong and I will really appreciate every kind of help.
With best regards
This is a bit of an odd issue. I realized that the new mkl version breaks opencv-python
. This issue was previously opened in the opencv-python
repo (opencv/opencv-python#308 (comment)), but I was suggested to opened it elsewhere. Hope you can provide some help or point me in the right direction.
OpenCV can load openh264-1.8.0-win64.dll
OpenCV cannot load openh264-1.8.0-win64.dll
Create a new conda environment with the following
name: test_opencv
channels:
- defaults
dependencies:
- python
- mkl==2020.0
- numpy
- pip
- pip:
- opencv-contrib-python
and run the following code:
import cv2
fourcc = cv2.VideoWriter_fourcc(*'X264')
out = cv2.VideoWriter('output.avi',fourcc, 20.0, (640,480))
By conda installing mkl==2020.0
opencv cannot load openh264-1.8.0-win64.dll
. The following error appears:
...
Failed to load OpenH264 library: openh264-1.8.0-win64.dll
Please check environment and/or download library: https://github.com/cisco/openh264/releases
...
Instead, installing mkl==2019.*
via conda the openH264 is loaded successfully:
...
OpenH264 Video Codec provided by Cisco Systems, Inc.
...
Per https://software.intel.com/en-us/node/522114 setting 0 for a number of threads is interpreted as setting it to "the default number of threads for the OpenMP run-time library."
Yet:
$ python -c "import mkl; mkl.domain_set_num_threads(0, domain='fft')"
-c:1: UserWarning: Non-positive argument of domain_set_num_threads is being ignored, number of threads will not be changed
I'm trying to install pypardiso
on Python 3.9, which depends on mkl-service
, but pip
complains that:
$ pip install pypardiso==0.3.3
Collecting pypardiso==0.3.3
Using cached pypardiso-0.3.3-py3-none-any.whl (10 kB)
ERROR: Could not find a version that satisfies the requirement mkl-service (from pypardiso) (from versions: none)
ERROR: No matching distribution found for mkl-service
Here's how to reproduce the bug:
pip install mkl mkl-service
python -c "import mkl"
and here's the output:
Collecting mkl
Downloading mkl-2021.4.0-py2.py3-none-win_amd64.whl (228.5 MB)
Collecting mkl-service
Downloading mkl_service-2.4.0-0-cp37-cp37m-win_amd64.whl (43 kB)
Collecting tbb==2021.*
Downloading tbb-2021.4.0-py3-none-win_amd64.whl (268 kB)
Collecting intel-openmp==2021.*
Downloading intel_openmp-2021.4.0-py2.py3-none-win_amd64.whl (3.5 MB)
Collecting six
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: tbb, intel-openmp, six, mkl, mkl-service
Successfully installed intel-openmp-2021.4.0 mkl-2021.4.0 mkl-service-2.4.0 six-1.16.0 tbb-2021.4.0
WARNING: You are using pip version 21.3; however, version 21.3.1 is available.
You should consider upgrading via the 'c:\hostedtoolcache\windows\python\3.7.9\x64\python.exe -m pip install --upgrade pip' command.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\mkl\__init__.py", line 49, in <module>
from . import _mklinit
ImportError: DLL load failed: The specified module could not be found.
Error: Process completed with exit code 1.
Currently:
In [1]: import mkl
In [2]: mkl.vml_get_mode()
Out[2]: ('ha', 'default', 'default')
In [3]: mkl.vml_set_mode(*mkl.vml_get_mode())
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
ValueError: Variable: <default> not in {'on': 2621440, 'off': 1310720}
Exception ignored in: 'mkl._py_mkl_service.__mkl_str_to_int'
ValueError: Variable: <default> not in {'on': 2621440, 'off': 1310720}
Out[3]: ('ha', 'default', 'default')
Due to this few days I cannot use conda to install any packages (use mirrors but no effects), so could we install mkl-service in pip?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.