versilov / matrex Goto Github PK
View Code? Open in Web Editor NEWA blazing fast matrix library for Elixir/Erlang with C implementation using CBLAS.
Home Page: https://hexdocs.pm/matrex
License: BSD 3-Clause "New" or "Revised" License
A blazing fast matrix library for Elixir/Erlang with C implementation using CBLAS.
Home Page: https://hexdocs.pm/matrex
License: BSD 3-Clause "New" or "Revised" License
inspect
can be used in multiple occasions, not only the terminal, so it is best to not rely on :io.columns
and instead use the width
option given to opts
. So instead of:
def inspect(%Matrex{} = matrex, opts) do
{:ok, columns} = :io.columns()
Matrex.Inspect.do_inspect(matrex, columns, 21)
end
One should do:
def inspect(%Matrex{} = matrex, opts) do
columns = opts.width
Matrex.Inspect.do_inspect(matrex, columns, 21)
end
Notice that width
may be infinity. In this case you may want to handle that by setting a default value, such as 80 or whatever.
PS: note there is a Matrix.Inspect.inspect function but that is exactly the same as calling IO.inspect(matrex)
. :)
I think this code summarizes my problem:
iex(1)> [a|b]= [1,2]
[1, 2]
iex(2)> a
1
iex(3)> b
[2]
Note that b
is between brackets because is a list.
iex(4)> [a|b] = [Matrex.new([[1]])|Matrex.new([[2]])]
[#Matrex[1×1]
┌ ┐
│ 1.0 │
└ ┘ |
#Matrex[1×1]
┌ ┐
│ 2.0 │
└ ┘]
iex(5)> a
#Matrex[1×1]
┌ ┐
│ 1.0 │
└ ┘
iex(6)> b
#Matrex[1×1]
┌ ┐
│ 2.0 │
└ ┘
I was expecting b
to be a list, and equal to:
iex(6)> b
[#Matrex[1×1]
┌ ┐
│ 2.0 │
└ ┘]
(Note the extra brackets.)
Hi @versilov,
I'm trying to round off numbers in a matrix. It works when my matrix contains only numbers.
matrix = Matrex.new([[1.9893432, 2.989899]])
Matrex.apply(matrix, &Float.round(&1, 2))
#Matrex[1×2]
┌ ┐
│ 1.99 2.99 │
└ ┘
However, the same function breaks when the matrix contains NaN's/Infinity.
Matrex.new([[1.9893432, 2.989899]])
|> Matrex.divide(0)
|> Matrex.apply(&Float.round(&1, 2))
** (FunctionClauseError) no function clause matching in Matrex.apply_on_matrix/3
The following arguments were given to Matrex.apply_on_matrix/3:
# 1
<<0, 0, 128, 127, 0, 0, 128, 127>>
# 2
#Function<6.128620087/1 in :erl_eval.expr/5>
# 3
<<1, 0, 0, 0, 2, 0, 0, 0>>
Attempted function clauses (showing 2 out of 2):
defp apply_on_matrix(<<>>, _, accumulator)
defp apply_on_matrix(<<value::float()-little()-size(32), rest::binary()>>, function, accumulator)
(matrex) lib/matrex.ex:693: Matrex.apply_on_matrix/3
(matrex) lib/matrex.ex:676: Matrex.apply/2
I've also tried using the Matrex.apply
function on each element as follows:
Matrex.new([[1.9893432, 2.989899]])
|> Matrex.divide(0)
|> Matrex.apply(fn val, _row, _col -> Float.round(val, 2) end)
** (FunctionClauseError) no function clause matching in Matrex.apply_on_matrix/6
The following arguments were given to Matrex.apply_on_matrix/6:
# 1
<<0, 0, 128, 127, 0, 0, 128, 127>>
# 2
#Function<18.128620087/3 in :erl_eval.expr/5>
# 3
1
# 4
1
# 5
2
# 6
<<1, 0, 0, 0, 2, 0, 0, 0>>
Attempted function clauses (showing 2 out of 2):
defp apply_on_matrix(<<>>, _, _, _, _, accumulator)
defp apply_on_matrix(<<value::float()-little()-size(32), rest::binary()>>, function, row_index, column_index, columns, accumulator)
(matrex) lib/matrex.ex:721: Matrex.apply_on_matrix/6
(matrex) lib/matrex.ex:690: Matrex.apply/2
I've also tried the array
branch for the same:
Matrex.new([[1.9893432, 2.989899]])
|> Matrex.divide(0)
|> Matrex.apply(fn val, index -> Float.round(val, 2) end)
** (FunctionClauseError) no function clause matching in Matrex.apply_on_matrix_float32/5
The following arguments were given to Matrex.apply_on_matrix_float32/5:
# 1
<<0, 0, 128, 127, 0, 0, 128, 127>>
# 2
#Function<12.128620087/2 in :erl_eval.expr/5>
# 3
{1, 1}
# 4
{1, 2}
# 5
""
Attempted function clauses (showing 2 out of 2):
def apply_on_matrix_float32(<<>>, _, _, _, accumulator)
def apply_on_matrix_float32(<<value::float()-little()-size(32), rest::binary()>>, function, pos, shape, accumulator)
(matrex) lib/matrex.ex:1833: Matrex.apply_on_matrix_float32/5
(matrex) lib/matrex.ex:759: Matrex.apply/2
Using a custom round function and checking for the type of each element also doesn't seem to work
defmodule Custom do
def round(value, precision) when is_float(value), do: Float.round(value, precision)
def round(value, _precision) do
value
end
end
Matrex.new([[1.9893432, 2.989899]])
|> Matrex.divide(0)
|> Matrex.apply(fn val, index -> Custom.round(val, 2) end)
** (FunctionClauseError) no function clause matching in Matrex.apply_on_matrix_float32/5
The following arguments were given to Matrex.apply_on_matrix_float32/5:
# 1
<<0, 0, 128, 127, 0, 0, 128, 127>>
# 2
#Function<12.128620087/2 in :erl_eval.expr/5>
# 3
{1, 1}
# 4
{1, 2}
# 5
""
Attempted function clauses (showing 2 out of 2):
def apply_on_matrix_float32(<<>>, _, _, _, accumulator)
def apply_on_matrix_float32(<<value::float()-little()-size(32), rest::binary()>>, function, pos, shape, accumulator)
(matrex) lib/matrex.ex:1833: Matrex.apply_on_matrix_float32/5
(matrex) lib/matrex.ex:759: Matrex.apply/2
Please let me know if I've missed out on any details! Thanks!
on arch with openblas, compilation fails with the following error:
a5% mix compile
==> matrex
Compiling: native/src/matrix_dot.c
native/src/matrix_dot.c:5:10: fatal error: cblas.h: No such file or directory
5 | #include <cblas.h>
| ^~~~~~~~~
compilation terminated.
make: *** [Makefile:200: _build/obj/matrix_dot.o] Error 1
could not compile dependency :matrex, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile matrex --force", update it with "mix deps.update matrex" or clean it with "mix deps.clean matrex"
cblas.h exists at /usr/include/openblas/cblas.h
if i manually edit matrix_dot.c to have #include <openblas/cblas.h>
instead, compilation succeeds
iex(24)> Matrex.new([[4_000_022.6578]])
#Matrex[1×1]
┌ ┐
│4000022.75 │
└ ┘
Passing a fairly large number causes the number to be rounded off - this gives me wrong values when I perform operations on these matrices.
Is this a known limitation? Are there any work-arounds for this to work with larger numbers?
Thanks in advance.
/cc @goodhamgupta
I'm a complete newb, running an IElixir kernel in JupyterLab notebook, tried out the Boyle package matrex_samples found at Notebook with Boyle examples with usage of Matrex library
All works fine until I get to calling heatmap()
I followed instructions for Ubuntu and got no errors when Boyle compiled the Matrex package in it's environment.
Any help greatly appreciated.
Hi,
Thank you so much for this wonderful library!! Would it be possible to get the diagonal of a square matrix and apply exponential functions to it with the current implementation?
Brilliant library. Of course Numpy has a 2 decades of accumulated functionality on top, so there's still a lot of stuff python-side that I'd want to use.
How can Matrix talk to Numpy efficiently, via, say erlports? Erlports sends python back as follows:
Erlang/OTP 21 [erts-10.0.8] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]
Interactive Elixir (1.7.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> {:ok, pid} = :python.start()
{:ok, #PID<0.176.0>}
iex(2)> xx = :python.call(pid, :py, :eig, [10])
{{:"$erlport.opaque", :python,
<<128, 2, 99, 110, 117, 109, 112, 121, 46, 99, 111, 114, 101, 46, 109, 117,
108, 116, 105, 97, 114, 114, 97, 121, 10, 95, 114, 101, 99, 111, 110, 115,
116, 114, 117, 99, 116, 10, 113, 1, 99, 110, 117, 109, 112, 121, ...>>},
{:"$erlport.opaque", :python,
<<128, 2, 99, 110, 117, 109, 112, 121, 46, 99, 111, 114, 101, 46, 109, 117,
108, 116, 105, 97, 114, 114, 97, 121, 10, 95, 114, 101, 99, 111, 110, 115,
116, 114, 117, 99, 116, 10, 113, 1, 99, 110, 117, 109, 112, ...>>}}
iex(3)> xx = :python.call(pid, :py, :eig_msgpack, [10])
<<146, 133, 164, 116, 121, 112, 101, 164, 60, 99, 49, 54, 164, 107, 105, 110,
100, 160, 162, 110, 100, 195, 165, 115, 104, 97, 112, 101, 145, 10, 164, 100,
97, 116, 97, 218, 0, 160, 61, 85, 216, 39, 118, 191, 18, 64, 0, 0, 0, 0, ...>>
iex(4)> Msgpax.unpack!(xx)
[
%{
"data" => <<61, 85, 216, 39, 118, 191, 18, 64, 0, 0, 0, 0, 0, 0, 0, 0, 152,
188, 99, 203, 39, 212, 235, 191, 0, 0, 0, 0, 0, 0, 0, 0, 176, 49, 23, 6,
60, 177, 190, 63, 82, 69, 170, 45, 5, 196, 229, 63, ...>>,
"kind" => "",
"nd" => true,
"shape" => '\n',
"type" => "<c16"
},
%{
"data" => <<24, 3, 175, 142, 209, 167, 205, 191, 0, 0, 0, 0, 0, 0, 0, 0, 78,
49, 82, 243, 187, 186, 202, 191, 0, 0, 0, 0, 0, 0, 0, 0, 24, 28, 80, 170,
100, 115, 198, 191, 68, 9, 19, 108, 248, 3, 178, ...>>,
"kind" => "",
"nd" => true,
"shape" => '\n\n',
"type" => "<c16"
}
]
As you can see, calling eig, which doesn't use msgpack, just returns a binary blob for Numpy. However if we msgpack the numpy arrays first, then we get a more structured return, which might be useful. The "data" field could go into a matrex matrix? How would one go about doing that?
Here by the way is the Python code:
from __future__ import print_function
import numpy as np
import pdb
import IPython
import string
import msgpack
import msgpack_numpy as m
m.patch() # patch msgpack to do numpy
def eig(n):
np.random.seed(8472)
xx = np.random.rand(n * n).reshape(n, n)
yy = np.linalg.eig(xx)
return yy
def eig_msgpack(n):
return msgpack.packb(eig(n))
def dicadd(dict):
return {string.join(dict.keys()): sum(dict.values())}
And here are my mix.exs deps:
defp deps do
[
{:erlport, "~> 0.10.0"},
{:benchwarmer, "~> 0.0.2"},
{:msgpax, "~> 2.0"}
]
Please tag the releases in github with "v#{@version}"
before/after mix hex.publish
, using:
git tag v0.6.7
git push --tags
hexdoc
automatically creates links on the right side of modules/functions headers ("</>"
) referencing the current version and without tags these links are broken.
Awesome work!
Would you mind benching against my library?
https://www.hex.pm/packages/graphmath
We have somewhat different use cases, but I'd be curious to see how pure elixir stands up for the things tested. Thank you!
I suggest deleting the boobs function. Other then a vessel for joke delivery
but this is nonsense.
@doc """
Function of a surface with two hills.
"""
@spec boobs(float, float) :: float
def boobs(x, y) do
x = (x - 40) / 4
y = (y - 40) / 4
:math.exp(-:math.pow(:math.pow(x - 4, 2) + :math.pow(y - 4, 2), 2) / 1000) +
:math.exp(-:math.pow(:math.pow(x + 4, 2) + :math.pow(y + 4, 2), 2) / 1000) +
0.1 * :math.exp(-:math.pow(:math.pow(x + 4, 2) + :math.pow(y + 4, 2), 2)) +
0.1 * :math.exp(-:math.pow(:math.pow(x - 4, 2) + :math.pow(y - 4, 2), 2))
end
I like jokes probably more than the next person unless they offend or cause
potential harm. Also I am just as guilty for this kind of thing, because we all
are idiots. MISTAKES MADE. LESSONS LEARN. WE GROW, IF GIVEN ROOM.
Formally and moreover:
... this form of code and practice may cause potential harm to
someall
participants in OSS. In a larger overarching context, condones and perpetuates
this insidious behavior in a larger community. Dudes, basically ...
Not my intention to be self-righteous judge and high-horse executioner. But there
going to be a large audience that'll be reading this (long and overly complex story).
They may not be so lenient in thought and/or action. Bottom-line, better to save
your ass. So to speak.
I am using Matrex in the context of AoC for the second year now, and it is not the first time I am confronted with the need to get the neighbours of a given position in the matrix.
That is, given a valid position {x,y}
, calculating the list of [{x-1, y-1}, {x-1, y}, {x-1, y+1}, {x, y-1}, {x, y+1}, {x+1, y-1}, {x+1, y}, {x+1, y+1}] (or a subset of this, including only valid points inside the bounds of the matrix, of course).
Would a PR implementing Matrex.neighbours/3
as
@spec neighbours(Matrix.t(), non_neg_integer(), non_neg_integer())
something that this project would be interested in? If so, I offer to implement such PR.
An alternative would be Matrex.neighbours/4
,
@spec neighbours(Matrix.t(), non_neg_integer(), non_neg_integer(), boolean())
with an optional argument (default true) to consider the diagonals (or not).
Right now the library uses "regular" NIFs. In general, for playing nice with the soft-realtime guarantees of the VM regular NIFs should execute in under 1ms. When they execute longer, it might throw off the load balancing of the schedulers (OS threads actually executing Erlang code) and lead to what is known as "scheduler collapse".
Fortunately, there are tools to prevent that.
If the job can be easily chunked the best solution is to periodically call enif_consume_timeslice
to check if code should yield and call enif_schedule_nif
to yield and allow the VM to execute other tasks, if necessary.
If chunking is not an option, the NIFs can be marked to execute on "dirty" schedulers - separate OS threads where execution can take however long it can and does not affect how regular schedulers work. The downside is that it incurs a context switch to a different OS thread - if the computation is complex enough, that overhead should not be noticeable.
After briefly looking at the code, it seems to be it would be good for the library to mark most functions as dirty (especially the element-wise operations or dot-product). A possible approach would also be to provide two implementations - dirty and not and call one of them depending on the size of the matrix. This might bring in more complexity then desired, though. An example of a library with a more intricate scheduler handling is enacl, which depending on the complexity runs functions on regular or dirty schedulers.
The atoms NaN
is represented internally as Elixir.NaN
that's why it is most commonly reserved for module names. The recommendation would be use explicit atoms such as: :nan | :inf | :neg_inf
or even :NaN | :Inf | :NegInf
.
@michalmuskala can you have a look at new structure of the Matrex.Array
object in this branch: https://github.com/versilov/matrex/blob/array/lib/matrex/array.ex
I've added types, arbitrary dimensions and strides. These are now stored in Elixir structure, and data
field contents only pure data, without dimensions. Exactly like you suggested in the post on ElixirForum.
I plan to merge this new structure into the Matrex structure.
Thanks in advance!
Foremost, thanks for this amazing project!
I noticed that on the benchmark section on the readme there are only comparisons between Elixir/Erlang libraries, which are not that fast/used.
Would be possible to benchmark against Numpy ? So we get a perspective against the most used library on this domain and how well we can expect matrex to perform?
The proposed solution to the error: native/src/matrix_dot.c:5:10: fatal error: 'cblas.h' file not found
is
open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg
however, that directory does not exist on macOS 10.15. The solution here did work for me.
export CPATH=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/
Honestly though this is just the first working result on google, not sure if it makes sense to add to the README or not.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.