External library and optimisation flags about fortran_magic

[mgaitan]

Hi @Vutshi . thanks for report it. It's right that -l and -L are not exposed to %%fortran. I could support them, but, just to learn, Do u know which would be the difference with the f2py's --link-* option (which is already available in the magic)?

[Vutshi]

Hi Martín,
I'm new to f2py so I might be wrong. As far as I understand --link-* works with something which is already exposed to the system (so numpy knows about it). -L and -l allows you to use your own library and don't mess with the system settings.

[mgaitan]

so, what do you think about to generalize in an --extra option to pass all not explicitly defined flags ?
for example

%%fortran  --extra ' -L/path_to_lib/lib -lopenblas -I/path/to/include/' 
<code>

Also it would be nice to pass -03 to f2py

this could be done with --opt

%%fortran --opt "-O3"

[Vutshi]

I think the --extra option is a good idea.
The --opt option is already occupied for me by --opt=-march=native. However, it is not a problem since --extra will take care about -O3 and anything else.

[mgaitan]

I've implemented this. Please, upgrade and thy this.

%%fortran  --extra ' -L/path_to_lib/lib -lopenblas -I/path/to/include/'  --opt "-march=native -O3"

let me know if everything is ok.
cheers
martin

[Vutshi]

Something is going wrong. I run the simple test:

%%fortran
subroutine f1(x, y, z)
    real, intent(in) :: x,y
    real, intent(out) :: z

    z = sin(x+y)

end subroutine f1

and get the error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-7c7c0006c542> in <module>()
----> 1 get_ipython().run_cell_magic(u'fortran', u'', u'subroutine f1(x, y, z)\n    real, intent(in) :: x,y\n    real, intent(out) :: z\n\n    z = sin(x+y)\n\nend subroutine f1')

/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_cell_magic(self, magic_name, line, cell)
   2127             magic_arg_s = self.var_expand(line, stack_depth)
   2128             with self.builtin_trap:
-> 2129                 result = fn(magic_arg_s, cell)
   2130             return result
   2131 

/Users/devasi/.ipython/extensions/fortranmagic.pyc in fortran(self, line, cell)

/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k)
    189     # but it's overkill for just that one bit of state.
    190     def magic_deco(arg):
--> 191         call = lambda f, *a, **k: f(*a, **k)
    192 
    193         if callable(arg):

/Users/devasi/.ipython/extensions/fortranmagic.pyc in fortran(self, line, cell)
    332                        verbosity=args.verbosity)
    333 
--> 334         self._code_cache[key] = module_name
    335         module = imp.load_dynamic(module_name, module_path)
    336         self._import_all(module, verbosity=args.verbosity)

TypeError: unhashable type: 'list'

[mgaitan]

ups. thats why we need tests.

ps: I`ll fix it in a moment
El 07/10/2013 11:48, "Denis Vasilyev" [email protected] escribió:

Something is going wrong. I run the simple test:

%%fortran
subroutine f1(x, y, z)
real, intent(in) :: x,y
real, intent(out) :: z

z = sin(x+y)

end subroutine f1

and get the error:

---

TypeError Traceback (most recent call
last)
in ()
----> 1 get_ipython().run_cell_magic(u'fortran', u'', u'subroutine f1(x,
y, z)\n real, intent(in) :: x,y\n real, intent(out) :: z\n\n z =
sin(x+y)\n\nend subroutine f1')

/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc
in run_cell_magic(self, magic_name, line, cell)
2127 magic_arg_s = self.var_expand(line, stack_depth)
2128 with self.builtin_trap:
-> 2129 result = fn(magic_arg_s, cell)
2130 return result
2131

/Users/devasi/.ipython/extensions/fortranmagic.pyc in fortran(self, line,
cell)

/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/IPython/core/magic.pyc
in (f, _a, *_k)
189 # but it's overkill for just that one bit of state.
190 def magic_deco(arg):
--> 191 call = lambda f, _a, *_k: f(_a, *_k)
192
193 if callable(arg):

/Users/devasi/.ipython/extensions/fortranmagic.pyc in fortran(self, line,
cell)
332 verbosity=args.verbosity)
333
--> 334 self._code_cache[key] = module_name
335 module = imp.load_dynamic(module_name, module_path)
336 self._import_all(module, verbosity=args.verbosity)

TypeError: unhashable type: 'list'

—
Reply to this email directly or view it on GitHub.

[mgaitan]

bumped 0.4.1

[Vutshi]

Hmmm something strange is going on here. Now everything works but not as expected :). The .so library compiled by the fortran magic doesn't contain link to the specified external library:
otool -L _fortran_magic_bb8a58e730478164cb30e999d1ef1ae7.so gives

_fortran_magic_bb8a58e730478164cb30e999d1ef1ae7.so:
    /usr/local/lib/libgfortran.3.dylib (compatibility version 4.0.0, current version 4.0.0)
    /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1669.0.0)
    /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/local/lib/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

but if I just copy the command generated by fortran magic and paste it into the terminal and run it then the library is properly compiled:

otool -L _fortran_magic_bb8a58e730478164cb30e999d1ef1ae7.so 
_fortran_magic_bb8a58e730478164cb30e999d1ef1ae7.so:
    /usr/local/lib/lib/libopenblas_sandybridgep-r0.2.8.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/lib/libgfortran.3.dylib (compatibility version 4.0.0, current version 4.0.0)
    /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1669.0.0)
    /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/local/lib/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

[mgaitan]

Could you share a notebook or a pastebin with a fully verbose example of your usage?

[Vutshi]

This is output of Fortran magic: http://pastebin.com/Z3qZ01fd
This is output of the same f2py command in terminal: http://pastebin.com/tKQ4xMMd
Here is the code to compile:

%%fortran -vvv --fcompiler=gnu95 --opt=-march=native  --extra '-L/usr/local/lib/lib -lopenblas'
subroutine matmul2 (a,b,c,a_dim1,a_dim2,b_dim1,b_dim2)
implicit none
real(8), dimension(a_dim1,a_dim2), intent(in) :: a
real(8), dimension(b_dim1,b_dim2), intent(in) :: b
real(8), dimension(b_dim1,a_dim2), intent(out) :: c
integer, intent(in) :: a_dim1,a_dim2,b_dim1,b_dim2

!call openblas_set_num_threads(1)

call dgemm('N', 'N', a_dim1, a_dim1, a_dim1, 1.0D0, a, a_dim1, b, a_dim1, 0.0D0, c, a_dim1)
end subroutine matmul2

[Vutshi]

It seems the problem is that the fortran magic provides -L"/usr/local/lib/lib -lopenblas" to the compiler

[mgaitan]

Thanks for debug it Denis.

I guess I patched it in bf644cf, but to avoid made a new broken release, could you test it before ?

Install that specific version with

%install_ext https://raw.github.com/mgaitan/fortran_magic/bf644cffa7da731439c762ff7b08d617290fc4a8/fortranmagic.py

[Vutshi]

Now it works like magic! Thanks a lot!

[mgaitan]

Great, thanks for your help. Released 0.4.2