Coder Social home page Coder Social logo

amnesia's Introduction

amnesia - mnesia wrapper for Elixir

amnesia wraps everything exposed by mnesia, from fragments to fragment hash, access and backup behaviors.

It provides a simplified table and database definition with some macros and allows you to use the nice Enum functions on tables by implementing the Enum.Iterator protocol.

Everything is documented and specced, even the unspecced and undocumented parts of mnesia that have been wrapped.

The documentation often refers to mnesia functions, I strongly suggest you read mnesia's documentation too, since it has a lot of valuable information.

Defining a database

To use amnesia you have to define a database and the tables of that database.

You can have multiple databases in the same amnesia instance, a database is actually just a way to group mnesia tables.

# needed to get defdatabase and other macros
use Amnesia

# defines a database called Database, it's basically a defmodule with
# some additional magic
defdatabase Database do
  # this is just a forward declaration of the table, otherwise you'd have
  # to fully scope User.read in Message functions
  deftable User

  # this defines a table with an user_id key and a content attribute, and
  # makes the table a bag; tables are basically records with a bunch of helpers
  deftable Message, [:user_id, :content], type: :bag do
    # this isn't required, but it's always nice to spec things
    @type t :: %Message{user_id: integer, content: String.t}

    # this defines a helper function to fetch the user from a Message record
    def user(self) do
      User.read(self.user_id)
    end

    # this does the same, but uses dirty operations
    def user!(self) do
      User.read!(self.user_id)
    end
  end

  # this defines a table with other attributes as ordered set, and defines an
  # additional index as email, this improves lookup operations
  deftable User, [{ :id, autoincrement }, :name, :email], type: :ordered_set, index: [:email] do
    # again not needed, but nice to have
    @type t :: %User{id: non_neg_integer, name: String.t, email: String.t}

    # this is a helper function to add a message to the user, using write
    # on the created records makes it write to the mnesia table
    def add_message(self, content) do
      %Message{user_id: self.id, content: content} |> Message.write
    end

    # like above, but again with dirty operations, the bang methods are used
    # thorough amnesia to be the dirty counterparts of the bang-less functions
    def add_message!(self, content) do
      %Message{user_id: self.id, content: content} |> Message.write!
    end

    # this is a helper to fetch all messages for the user
    def messages(self) do
      Message.read(self.id)
    end

    # like above, but with dirty operations
    def messages!(self) do
      Message.read!(self.id)
    end
  end
end

Creating the database

Before using a database you have to create it, and before it a schema.

To do so, you can use the built-in mix task amnesia.create passing your database module via the --database or -d options.

mix amnesia.create -d Database --disk

The available options for creating the databases are:

  • --database or -d: the database module to create
  • --no-schema: to avoid creating the schema
  • --memory: to create the tables with memory copying on the current node
  • --disk: to create the tables with disc_copies on the current node
  • --disk!: to create the tables with disc_only_copies on the current node

By default it creates the schema and uses disc_copies.

If you want to drop the tables there is also a drop task you should use with CAUTION as it will destroy all data. To use it just call:

mix amnesia.drop -d Database

The options accepted by this task are:

  • --database or -d: same as with create. A database module to drop tables
  • --schema: drops the schema too. Defaults to false

Writing to the database

Once the database has been defined and created, you can start using the various tables.

# You want to be in a transaction most of the time, this ensures the data
# doesn't get corrupted and you get meaningful values back.
#
# Most operations won't work outside a transaction and will raise an exception.
Amnesia.transaction do
  # Every table is a record, so you can do everything you can do with records.
  #
  # Once you want to save the record, you have to call `.write` on it, this
  # will write the record to the table.
  #
  # Since we defined the `User` table with an `autoincrement` id attribute it
  # will be incremented internally on write, unless the id attribute is set, in
  # that case it will be left as is.
  #
  # If you want to know the values of the autoincrement fields, `.write` always
  # returns the updated record.
  john = %User{name: "John", email: "[email protected]"} |> User.write

  # Let's create more users.
  richard = %User{name: "Richard", email: "[email protected]"} |> User.write
  linus   = %User{name: "Linus", email: "[email protected]"} |> User.write

  # Now let's add some messages.

  john |> User.add_message %S"""
  When we program a computer to make choices intelligently after determining
  its options, examining their consequences, and deciding which is most
  favorable or most moral or whatever, we must program it to take an attitude
  towards its freedom of choice essentially isomorphic to that which a human
  must take to his own.
  """

  john |> User.add_message %S"""
  He who refuses to do arithmetic is doomed to talk nonsense."
  """

  john |> User.add_message %S"""
  It's difficult to be rigorous about whether a machine really 'knows',
  'thinks', etc., because we're hard put to define these things. We understand
  human mental processes only slightly better than a fish understands swimming.
  """

  richard |> User.add_message %S"""
  For personal reasons, I do not browse the web from my computer. (I also have
  no net connection much of the time.) To look at page I send mail to a daemon
  which runs wget and mails the page back to me. It is very efficient use of my
  time, but it is slow in real time.
  """

  richard |> User.add_message %S"""
  I am skeptical of the claim that voluntarily pedophilia harms children. The
  arguments that it causes harm seem to be based on cases which aren't
  voluntary, which are then stretched by parents who are horrified by the idea
  that their little baby is maturing.
  """

  linus |> User.add_message %S"""
  Portability is for people who cannot write new programs.
  """

  linus |> User.add_message %S"""
  Really, I'm not out to destroy Microsoft. That will just be a completely
  unintentional side effect.
  """

  linus |> User.add_message %S"""
  Modern PCs are horrible. ACPI is a complete design disaster in every way. But
  we're kind of stuck with it. If any Intel people are listening to this and
  you had anything to do with ACPI, shoot yourself now, before you reproduce.
  """
end

Reading from the database

Once there's something written to the database you can start reading back records from it.

Amnesia.transaction do
  # The simplest way to read a record is using the key of the record (by
  # default the first attribute)
  #
  # Since we wrote the John, Richard and Linus in this order and the id is
  # defined as *autoincrement*, the first `User` will be John.
  john = User.read(1)

  # Now let's read his messages and print them all.
  john |> User.messages |> Enum.each &IO.puts(&1.content)

  # You can also use an Exquisite selector to fetch records.
  selection = Message.where user_id == 1 or user_id == 2,
    select: content

  # Get the values in the selector and print them.
  selection |> Amnesia.Selection.values |> Enum.each &IO.puts(&1.content)
end

Other documentation

All the code has @spec and @doc, so you can either go around the source and read the @docs, use the REPL and h/1 or generate the documentation with ex_doc.

amnesia's People

Contributors

benwilson512 avatar briksoftware avatar cjimison avatar connorrigby avatar cs-victor-nascimento avatar derkastellan avatar ijcd avatar jlholm avatar macabeus avatar matehat avatar meh avatar mweibel avatar nazarsh avatar optikfluffel avatar rodrigues avatar samterrell avatar sheharyarn avatar zorbash avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

amnesia's Issues

Pass arguments to deftable

I'm trying to set the attributes of a table from a variable / module_attribute in the deftable macro, but it raises an exception instead.

use Amnesia

defdatabase MyApp.DB do
  @attrs [:id, :a, :b, :c]

  deftable SomeTable, @attrs, type: :bag do
  end 
end

I also tried using unquote(@attrs) since it's a macro but that doesn't work either. Is there some way I can get this to work? (This is just an example, the attributes need to be loaded from another module during compile-time).

This is the error I get:

Compiling 1 file (.ex)

== Compilation error on file lib/mnesia.ex ==
** (ArgumentError) argument error
    :erlang.length({:@, [line: 6], [{:attrs, [line: 6], nil}]})
    lib/amnesia/table/definition.ex:63: Amnesia.Table.Definition.define/4
    expanding macro: Amnesia.Database.deftable/4
    lib/db.ex:6: MyApp.DB (module)

Cannot compile dependency from mix.exs

Im getting this error when I try to include this lib as a dependency:

** (Mix) No package with the name amnesia in registry

Here is my markup:

defp deps do
[
{:amnesia, "~> 0.1.1"},
{:collector, in_umbrella: true}
]
end

What am I doing wrong or how should I include it in my project manually

-d does not work in amnesia.drop/amnesia.create

# mix amnesia.drop -d Database
** (FunctionClauseError) no function clause matching in Mix.Amnesia.ensure_database_module/1
    lib/mix/mix_amnesia.ex:10: Mix.Amnesia.ensure_database_module(nil)
    lib/mix/amnesia.drop.ex:10: Mix.Tasks.Amnesia.Drop.run/1
    (mix) lib/mix/task.ex:296: Mix.Task.run_task/3
    (mix) lib/mix/cli.ex:58: Mix.CLI.run_task/2
    (elixir) lib/code.ex:363: Code.require_file/2

There's no error if I replace -d with --database. I'm using amnesia 0.2.5, mix 1.3.3.

running amnesia in different environments, tests isolation

Good day,

I am new to Elixir and erlang, I am now playing with amnesia in phoenix application. I am unsure how to setup testing environment in phoenix application with amnesia, so that it would be as smooth as with ecto (or at least close to). How do you usually go about setting things up in phoenix with amnesia?

compilation issue

== Compilation error in file lib/exquisite.ex ==
** (CompileError) lib/exquisite.ex:103: CALLER is available only inside defmacro and defmacrop
lib/exquisite.ex:103: (module)
could not compile dependency :exquisite, "mix compile" failed. You can recompile this dependency with "mix deps.compile exquisite", update it with "mix deps.update exquisite" or clean it with "mix deps.clean exquisite"

Elixir Version = Elixir v1.7.0-dev

(UndefinedFunctionError) function :erl_eval.match1/4 is undefined or private

When I try to use example code in IEx

Amnesia.transaction do
john = %User{name: "John", email: "[email protected]"} |> User.write
end

error is raised.

** (UndefinedFunctionError) function :erl_eval.match1/4 is undefined or private
       (uym) User.write(%User{email: "[email protected]", id: nil, name: "John"})
    (stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
    (stdlib) erl_eval.erl:438: :erl_eval.expr/5
    (mnesia) mnesia_tm.erl:835: :mnesia_tm.apply_fun/3
    (mnesia) mnesia_tm.erl:810: :mnesia_tm.execute_transaction/5

in lib/database.ex is example code from README. Database was created using

mix amnesia.create -db Database --disk

Content of database.ex

use Amnesia
defdatabase Database do
  deftable User

  deftable Message, [:user_id, :content], type: :bag do
    @type t :: %Message{user_id: integer, content: String.t}

    def user(self) do
      User.read(self.user_id)
    end

    def user!(self) do
      User.read!(self.user_id)
    end
  end

  deftable User, [{ :id, autoincrement }, :name, :email], type: :ordered_set, index: [:email] do
    @type t :: %User{id: non_neg_integer, name: String.t, email: String.t}

    def add_message(self, content) do
      %Message{user_id: self.id, content: content} |> Message.write
    end

    def add_message!(self, content) do
      %Message{user_id: self.id, content: content} |> Message.write!
    end

    def messages(self) do
      Message.read(self.id)
    end

    def messages!(self) do
      Message.read!(self.id)
    end
  end
end

defmodule User do
  defstruct id: nil, name: nil, email: nil
end

defmodule Message do
  defstruct user_id: nil, content: nil
end

I use Elixir 1.3.2, {:amnesia, "~> 0.2.4"}.

What could be wrong?

How to create a table for existing database

I'm trying to create a table in an existing database using exrm's release tasks. How is everyone else doing this? I'm doing it the following way:

def create_table(production_node, database, table_name) do
  {:ok, _} = Application.ensure_all_started(:app)

  # create table
  table_name.create(disk: [production_node])

  IO.puts "Waiting"
  :ok = database.wait(15000)

  IO.puts "Done"
  :init.stop()
end

Could we perhaps put something like this in the readme? Assuming this is correct, should I send a PR?

Warnings with latest and Elixir 1.1.1

Just upgraded to elixir 1.1.1 did a mix deps.update amnesia and now I am seeing the following warning

==> exquisite
warning: the dependency :exquisite requires Elixir "~> 1.0.0-rc1" but you are running on v1.1.1

Everything is working, just though I would report this.

Always get no transaction even though I have a transaction

Based on looking at the tests I wrote:

Amnesia.transation! do
Enum.each quotes, fn(q) ->
StockData.StockQuotes[symbol: q.symbol, date: q.date, open: q.open, high: q.high,
low: q.low, close: q.close, volume: q.volume, adjClose: q.adjClose].write
end
end

And it always fails with:

** (exit) {:aborted,:no_transaction}
mnesia.erl:309: :mnesia.abort/1
lists.erl:1313: :lists.foreach/2
/Users/gclements/work/elixir/lib/elixir/lib/enum.ex:284: Enum.each/2

Amnesia.start and StockData.StockQuotes.create both succeeded. What am I missing?

Dynamic table names with `deftable` aren't possible

Hello!

I'm trying to create bunch of tables at once with the following code:

use Amnesia

defdatabase Database do
  for name <- [:"Elixir.TestTable", :"Elixir.AnotherTestTable"] do
    deftable name, [{ :id, autoincrement }, :user_id], type: :set do
    end
  end
end

That gives me the following error:

== Compilation error on file web/models/database.ex ==
** (CompileError) web/models/database.ex:5: undefined function name/0
    (stdlib) lists.erl:1337: :lists.foreach/2
    web/models/database.ex:5: anonymous fn/2 in :elixir_compiler_2.__MODULE__/1
    (elixir) lib/enum.ex:1473: Enum."-reduce/3-lists^foldl/2-0-"/3
    web/models/database.ex:4: (module)
    (elixir) lib/kernel/parallel_compiler.ex:100: anonymous fn/4 in Kernel.ParallelCompiler.spawn_compilers/8

I noticed that this code, however, is compiled correctly:

defdatabase Database do
  for name <- [:"Elixir.TestTable", :"Elixir.AnotherTestTable"] do
    defmodule name, do: []
  end
end

Given that, I decided to ask that question in the Amnesia tracker, not in some general Elixir thread.

How do I do table definitions with dynamic names correctly?

Compiling with Elixir 1.3 give the following warnings

warning: function Amnesia.result/1 is undefined or private
Found at 7 locations:
lib/amnesia.ex:159
lib/amnesia.ex:168
lib/amnesia.ex:194
lib/amnesia.ex:204
lib/amnesia.ex:229
lib/amnesia.ex:255
lib/amnesia.ex:281

Making a disk based DB?

Sorry, this is more of a question than an issue. How do you make a disk based database? The RAM based code works fine. I'd just like to persist data to disk.

I think it should be:

Stocks.StockData.create [{:disk, [node]}]
Stocks.StockData.wait

but it hangs forever in wait. Am I calling create wrong?

[FEATURE] - Default Mix tasks

I think it would help a lot to have some default mix tasks like create|drop|migrate (same as those found in ecto for easier understanding). Your example of install/uninstall in the README is pretty much what everybody does when starting a project with amnesia INMHO.

We would only need to support something like:

mix amnesia.create --database MyDatabase (more options)

Options I can think of are:

  • --no-schema: would not create the mnesia schema
  • --disc: would create passing the option disc:[node()]
  • --disc-only: same as above but with disc only on node
    ... and other mnesia options.

To support migrate we could have some behaviour that would export a migrate function and we would receive the user module implementing that function. Again, close to Ecto's idiom.

I was starting out a PR but thought that it would be better to ask for opinions first. What do you think @meh?

delete() not working on a table

When I do MyTableName.delete(MyTableName, "mykey") I get "** (CaseClauseError) no case clause matching: "mykey"". Am I passing the name parameter wrong? The documentation basically just says, "see mnesia:delete". The mnesia:delete documentation says to pass the name of the table, but that seems unnecessary, since I've already specified the table with MyTableName.delete(). I like your library (and it's license terms) and I'd like to use it, but I'm really stuck here.

amnesia no longer compiles under v0.9.x of Elixir.

amnesia no longer compiles under v0.9.x of Elixir.

Enum.Iterable no longer exists in Elixir. It was removed in v0.9.0. It was replaced by Enumerable.

Also there are a few places where the code references List.Dict.keys. It should be Dict.keys.

But the most work is in Amnesia.Table.Iterator. It looks like iterate needs to be replaced with reduce.

How to use Exquisite selector?

Hi,

I'm having an issue using the exquisite selector. I've written this code, from the README section:

    Amnesia.transaction do
      existing_user = User.where(email == new_mail, select: id) |> Amnesia.Selection.values
      existing_user
    end

My table user: deftable User, [ :id, :email, :name, :password ], type: :ordered_set, index: [:email]

At compilation time i'm getting this error:

== Compilation error on file lib/mnesia.ex ==
** (CompileError) lib/mnesia.ex:51: function email/0 undefined
    (stdlib) lists.erl:1336: :lists.foreach/2
    lib/mnesia.ex:31: (module)

What I am missing? Thanks!

Issue with Table object

Hi,

Attempting to create an app using the examples on the README, I get the following error:

** (CompileError) lib/test/connection_store.ex:67: User.struct/0 is undefined, cannot expand struct User
(elixir) src/elixir_map.erl:44: :elixir_map.translate_struct/4
(stdlib) lists.erl:1352: :lists.mapfoldl/3
(elixir) src/elixir_clauses.erl:36: :elixir_clauses.clause/7
(elixir) src/elixir_fn.erl:9: anonymous fn/3 in :elixir_fn.translate/3
(stdlib) lists.erl:1352: :lists.mapfoldl/3

Is Amnesia supported in Elixir 0.15.0?

Thanks

Amnesia fails to compile with Elixir 1.0.5, Erlang 18

== Compilation error on file lib/amnesia/event.ex ==
** (CompileError) lib/amnesia/event.ex:29: type record() undefined
(stdlib) lists.erl:1337: :lists.foreach/2
(stdlib) erl_eval.erl:669: :erl_eval.do_apply/6

deftable: warning: the guard for this clause evaluates to 'false'

Hi,

thanks a lot for your effort to update it to the latest elixir version.
Since updating however, I have the mentioned warning when trying to compile my table definitions.

A sample definition where this comes up is the following:

#...
deftable Client, [:ref, :user_id, :data, :is_admin], type: :set, index: [:user_id], copying: :memory do
#...

Do you have an idea why this warning shows up?

Can't works in Elixir v0.14.1

I used this project in Elixir 0.14.1, and my module looks like:

defmodule Cc do
  use Amnesia

  defdatabase Database do
    deftable Job, [{:id, autoincrement}, :mod, :vnode, :queue, :status, :opts], type: :ordered_set do
    end

    deftable Hint, [{:id, autoincrement}, :vnode, :url], type: :set do
    end

    def start() do
      Amnesia.Schema.create

      # start mnesia
      Amnesia.start

      # create tables
      #Clear.Database.Job.create(disk: [node])
      #Clear.Database.Hint.create(disk: [node])
      Database.create(disk: [node])

      # This waits for the database to be fully created.
      Database.wait
    end

    def stop do
      Amnesia.stop
    end

  end
end

in iex, I start Amnesia, then I want to create a new column,

iex> Cc.Database.Job[mod: :mod1, vnode: 1, queue: nil, status: :new, opts: []]
** (Protocol.UndefinedError) protocol Access not implemented for Cc.Database.Job, only the nil atom is supported
    (elixir) lib/access.ex:144: Access.Atom.undefined/1

I think it results of the Record isn't belongs to Elixir Kernel anymore, could you fix it to adapt Elixir's Map struct?

Database.create can't handle disk only storage?

How can I start Amnesia in disk_only_mode ?

I tried:

Database.create disk!: [node]

with result:

Database creation failure: [error: {:bad_type, Database, :disc_only_copies, :"smalldev@MacBook-Air"}, error: {:bad_type, Database.User, :disc_only_copies, :"smalldev@MacBook-Air"}, error: {:bad_type, Database.History, :disc_only_copies, :"smalldev@MacBook-Air"}]

What i'm doing wrong?

"Backup restore" doesn't restore the last value of "autoincrement" column

I'm using backup commands to create a snapshot of my database.
To create a snapshot, I'm using this code:

Amnesia.Backup.checkpoint(%{
  name: 'backup',
  max: [Database.Account, Database.VerifyCode],
  override: true
})
Amnesia.Backup.start('backup', 'backup')

It's works, creating a file called backup. But, when I restore this snapshot using

Amnesia.Backup.restore('backup', nil)

The tables that has an autoincrement column get problem, because the restore command doesn't restore the counter.
For example, if I restore the Account table, I get 5 rows from backup file, then, if I to add a new record on this table, the new record will replace the first record (because will have the same id, 1, instead of the correct id, 5).

Version in mix.exs is still 0.0.1

Should be "0.0.2" for repository tag "v0.0.2". Otherwise the dependency given on http://expm.co results in the following error message:

Unchecked dependencies for environment dev:

  • amnesia [git: "https://github.com/meh/amnesia.git"]
    the dependency does not match the specified version, got 0.0.1

License update

Is there any chance you would be considering changing the chosen license?

I work in a start up where we are adopting Elixir and would love to use this package, but I see two major problems:

  1. Your license terms basically allow me to wholesale steal your code without giving you credit. Obviously I don't want to do this because it's not just a jerk move, I consider it bad open source form to not give credit to the guy who did all the heavy lifting.
  2. We integrate with enterprise customers, and as much as I love the ideals behind the license, there's no way I can convince to my boss or our clients to adopt software filled with a bunch of f-bombs, no matter how awesome. As stupid a thing as this is to prevent the usage of awesome software, corporate attitudes can hurt adoption.

Any idea if this could be changed to an equally permissive license which also gives you more credit while not horrifying a bunch of stuffed shirts?

Amnesia does not compile with Elixir 0.14.3

I get this error with Elixir 0.14.3. Downgrading to 0.14.2 solves the issue.

= Compilation error on file lib/amnesia/metadata.ex ==
** (CompileError) lib/amnesia/metadata.ex:105: type t() undefined
    (stdlib) lists.erl:1336: :lists.foreach/2
    (stdlib) erl_eval.erl:657: :erl_eval.do_apply/6
    (elixir) src/elixir.erl:170: :elixir.erl_eval/3
    (elixir) src/elixir.erl:158: :elixir.eval_forms/4
    (elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/2
    (elixir) src/elixir.erl:170: :elixir.erl_eval/3

Full configuration: Elixir 0.14.3 and Erlang R17.1 on MacOS X 10.9.4.

[Request] Raw pattern matching support or raw wrapping support

To avoid the need for multiple index rows it would be ideal to be able to perform partial pattern matching on tuples

e.g.

:mnesia.match_object({AMNESIA.TABLE, {:tuple, :encoded, :cartesian, :index, :_})

But handling this would either require writing custom exquisite queries (? maybe)

or abandoning Amnesia.

It should be possible to provide a sort of last resort filtering library such that

Amnesia.Fragment.transaction do
  Amnesia.wrapper do
    :mnesia.match_object({AMNESIA.TABLE, {:tuple, :encoded, :cartesian, :index, :_})
  end
end

where the wrapper logic determines if the inner term returns

{[values], continuation} (:mnesia.select) or [values] (:mnesia.match_object) or %end_of_table or RecordList

and populates the appropriate Amnesia.Table.Select{coerce: table_from_result, continuation: continuation_from_result, values: values_from_result} or appropriate error response.

Some sort of built in last resort macro or documentation for performing partial matches if possible directly using Amnesia/Exquisite would greatly improve the portability of Amnesia.

I'm happy to provide the code for such if the code maintainers have any guidelines, requirements for such a change, but I'd need guidance on any pure Amnesia solution here.

Queries with and/or operators not working (on Elixir 1.6)

On Elixir 1.6.0, where statements with and or or operators cause compilation errors. The where macro works fine when they are not used, but when they are, for some reason elixir expands the match fields to functions (which don't exist), throwing a compilation error.

Environment Details:

  • Elixir 1.6.0
  • Erlang 20.1
  • Amnesia 0.2.7 (Exquisite 0.1.7)
  • MacOS High Sierra 10.13.2

Reproducing the Issue:

A simple elixir application with amnesia set up, along with this DB/Table. This would not compile on Elixir 1.6.0, but if you remove the find/2 method, it will.

use Amnesia

defdatabase DB do
  deftable Post, [:user, :status, :content] do

    # Insert a Post
    def insert(user, status, content) do
      Amnesia.transaction do
        write(%DB.Post{user: user, status: status, content: content})
      end
    end


    # Find all posts by user
    def find(p_user) do
      Amnesia.transaction do
        where(user == p_user)
        |> Amnesia.Selection.values
      end
    end


    # Find all posts by user and status
    def find(p_user, p_status) do
      Amnesia.transaction do
        where(user == p_user and status == p_status)
        |> Amnesia.Selection.values
      end
    end

  end
end

Error:

Erlang/OTP 20 [erts-9.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

Compiling 1 file (.ex)
warning: variable "user" does not exist and is being expanded to "user()", please use parentheses to remove the ambiguity or change the variable name
  lib/amnesia.ex:26

warning: variable "status" does not exist and is being expanded to "status()", please use parentheses to remove the ambiguity or change the variable name
  lib/amnesia.ex:26


== Compilation error in file lib/amnesia.ex ==
** (CompileError) lib/amnesia.ex:26: undefined function status/0
    (stdlib) lists.erl:1338: :lists.foreach/2
    lib/amnesia.ex:4: (module)
    (elixir) lib/kernel/parallel_compiler.ex:198: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/6

Everything else works as expected. This is a serious bug which also affects one of my projects; Que. Hoping this gets resolved soon!

Fails to compile on fragment.ex

 mix deps.compile amnesia
* Compiling amnesia
Compiled lib/amnesia/counter.ex
Compiled lib/amnesia.ex
Compiled lib/amnesia/backup.ex
lib/amnesia/event.ex:43: partial application without capture is deprecated
Compiled lib/amnesia/access.ex
Compiled lib/amnesia/event.ex
Compiled lib/amnesia/database.ex
Compiled lib/amnesia/fragment/hash.ex
Compiled lib/amnesia/schema.ex
== Compilation error on file lib/amnesia/fragment.ex ==
could not compile dependency amnesia, mix compile failed. If you want to recompile this dependency, please run: mix deps.compile amnesia
** (SyntaxError) deps/amnesia/lib/amnesia/fragment.ex:77: unhandled operator ->
    src/elixir_translator.erl:600: :elixir_translator.translate_arg/2
    lists.erl:1339: :lists.mapfoldl/3
    lists.erl:1340: :lists.mapfoldl/3
    src/elixir_translator.erl:607: :elixir_translator.translate_args/2
    src/elixir_literal.erl:24: :elixir_literal.translate/2
    src/elixir_translator.erl:600: :elixir_translator.translate_arg/2
    lists.erl:1339: :lists.mapfoldl/3
    deps/amnesia/lib/amnesia/fragment.ex:77: Amnesia.Fragment.properties/1

Does read_at/indexes works?

I tried to create table with

deftable Test, [:id, :content, :secondary_index],
                     type: :ordered_set,
                     index: [:secondary_index] do
...

then when i tried to read by index i get error

Test.read_at(index_value, :secondary_index)

{:no_exists, Database.Test, {:index, [3]}}

:badarg when following the README

I've just copied this from the readme and put it in lib/Database.ex:

use Amnesia

defdatabase Database do
  deftable User, [{ :id, autoincrement }, :name, :email], type: :ordered_set, index: [:email] do
    # again not needed, but nice to have
    @type t :: %User{id: non_neg_integer, name: String.t, email: String.t}
  end
end

Then I created the database and tried reading / writing to the database, but it keeps returning :badarg:

➜  my_app git:(master) ✗ mix amnesia.create -db Database
Compiling 1 file (.ex)

20:15:37.891 [info]  Application mnesia exited: :stopped
➜  my_app git:(master) ✗ iex -S mix
Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Interactive Elixir (1.3.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> use Amnesia
Amnesia.Helper
iex(2)> use Database
[Amnesia, Amnesia.Fragment, Exquisite, Database, Database.User, Database.User]
iex(3)> Amnesia.transaction do
...(3)> john = %User{name: "John", email: "[email protected]"} |> User.write
...(3)>
...(3)> end
:badarg
iex(4)> Amnesia.transaction do
...(4)> User.read(1)
...(4)> end
:badarg
iex(5)>

What am I doing wrong?

BTW I also posted this question on stackoverflow: http://stackoverflow.com/questions/40965818/elixir-amnesia-badarg-when-doing-anything-to-the-database

Mix task to create database fails

Hi,

I am trying to create example application using Phoenix with Amnesia. But, having trouble creating the database using the mix task which fails with below error. I am not sure the docs that I found from blogs are up to date or I missed something.

mix amnesia.create -d Database --disk
** (Mix) could not load Database, error: :nofile. Please pass a proper database with the --database option.

mix --version
Erlang/OTP 20 [erts-9.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Mix 1.6.1 (compiled with OTP 20)

Example Git repo: https://github.com/ganimp84/amnesia_example

Fails to compile on table.ex

I see that there are couple other issues with compiling amnesia, but none of them helped solve me my problem:

== Compilation error on file lib/amnesia/table.ex ==
could not compile dependency amnesia, mix compile failed. If you want to recompile this dependency, please run: mix deps.compile amnesia
** (SyntaxError) deps/amnesia/lib/amnesia/table.ex:1569: syntax error before: '{'

I would like to use Amnesia in my Dynamo app. Here's my dependencies description:

  defp deps do
    [ { :cowboy, github: "extend/cowboy" },
      { :dynamo, "0.1.0-dev", github: "elixir-lang/dynamo" },
      { :amnesia, "0.1.1", [ github: "meh/amnesia", tag: "v0.1.1" ] } ]
  end

I use Elixir v0.10.3.

Amnesia 0.2.5 / Elixir 1.4.2 defdatabase warning (x4): variable "metadata" does not exist...

Using Amnesia 0.2.5 with Elixir 1.4.2, I see a warning repeated 4 times for every line where the defdatabase macro is called during compilation (via mix compile, if that makes a difference). Luckily, it's reproducible by simply making a new test.ex file with the following contents:

use Amnesia

defdatabase Test do
end

Compiling the above will result in the following, repeated 4 times for every use of defdatabase in each file being compiled:

warning: variable "metadata" does not exist and is being expanded to "metadata()", please use parentheses to remove the ambiguity or change the variable name

How to create the database on production machine without mix

I am using exrm to create my uat and production releases. They are created with all the compiled beams as well as the erts and tar'd and zipped into a standalone file that gets extracted on the target box. How am I supposed to create the initial mnesia db via amnesia (mix amnesia.create -db Database) on the target box?

Thanks

Geoff

Amnesia.Table.transform

When I run Amnesia.Table.transform/3 in the iex shell the update works fine. However when I put it into a Mix.Tasks module it doesn't. Any hints on using Amnesia.Table.transform/3 in a Task ?

The same issue occurs when I put a :mnesia.transform_table in the run/1 function of the Task.

Hangs on the Database.wait in "install" mix task

Hey there,

Thanks for the wrapper. I'm having trouble following samples. I defined the Database module, then Install / Uninstall tasks and after compiling and running "iex -S mix install" I can see that it hangs on the Database.wait line infinitely.

Any ideas what to try or where to look for help are greatly appreciated!

Message does not get deleted

For some reason delete does not actually delete a message. Here is a relevant test. You can skip first three lines related to phoenix, it is related to the code writing to amnesia.

test "server receives message, stores it and replies with `received` status", %{socket: socket, user: user, chat: chat} do
    {:ok, _, socket} = subscribe_and_join(socket, "chats:#{chat.id}", %{})
    ref = push socket, "message", %{body: "the body", client_id: 1, created_at: :os.system_time(:seconds)}
    assert_reply ref, :ok, %{client_id: 1, status: "received"}

    [message] = Amnesia.transaction do
      selection = Message.where chat_id == chat.id, select: [id, body, client_id, chat_id, created_at, status, user_id]
      selection
      |> Amnesia.Selection.values
    end
    chat_id = chat.id
    user_id = user.id
    assert [id, "the body", 1, ^chat_id, _, "received", ^user_id] = message

    # Trying to delete here
    status = Amnesia.transaction do
      IO.inspect Message.last
      Message.last |> Message.delete
    end

    IO.inspect status

    # receiving ** (MatchError) no match of right hand side value: [[1, "the body", 1, 293, 1465198234, "received", 1514]]
    [] = Amnesia.transaction do
      selection = Message.where chat_id == chat.id, select: [id, body, client_id, chat_id, created_at, status, user_id]
      selection
      |> Amnesia.Selection.values
    end
  end

function to_binary/1 undefined

Hi,

It doesnt compile on the last elixir ( 0.10.3 ) :(

mix deps.compile amnesia

  • Compiling amnesia
    Compiled lib/amnesia/counter.ex
    == Compilation error on file lib/amnesia.ex ==
    could not compile dependency amnesia, mix compile failed. If you want to recompile this dependency, please run: mix deps.compile amnesia
    ** (CompileError) deps/amnesia/lib/amnesia.ex:66: function to_binary/1 undefined
    lists.erl:1323: :lists.foreach/2
    /private/tmp/elixir-1HDh/elixir-0.10.3/lib/elixir/lib/kernel/parallel_compiler.ex:82: Kernel.ParallelCompiler."-spawn_compilers/8-fun-0-"/3

continuum fails to compile as a dependency

I'm having trouble compiling continuum with mix deps.compile (Elixir 0.12.4):

== Compilation error on file lib/date.ex ==
could not compile dependency continuum, mix compile failed. You can recompile this dependency with `mix deps.compile continuum` or update it with `mix deps.update continuum`
** (UndefinedFunctionError) undefined function: Timezone.__using__/1
    Timezone.__using__([])
    lib/date.ex:2: (module)
    (stdlib) erl_eval.erl:569: :erl_eval.do_apply/6
    (elixir) src/elixir.erl:140: :elixir.eval_forms/4

Strangely, compiling it independently works just fine. Any ideas?

Queries as functions within deftable don't seem to work

Hey,

I'm probably doing something wrong but after trying various times I can't get a working result.
I have various queries specified within the deftable block and they don't seem to run (I don't even get an error usually). Previously I worked around this issue using Exquisite directly but somehow this doesn't work anymore since the newest Elixir release.

Sample function:

deftable ClientStatus, [:user_id, :room_id, :status], type: :bag, copying: :disk do
    @type clientStatusTypes :: ClientStatus[user_id: String.t, room_id: binary, status: boolean]

    def is_user_admin?(id) do
        selector = where user_id == id and status == :admin,
                            select: status
        case selector do
            nil ->
                false
            _result ->
                true
        end
    end
end

Do you have an idea what might be wrong here? I also tried doing various other ways to achieve the same result but all of them don't work..

# Error on compilation: function status/0 undefined
selector = ClientStatus.where user_id == id and status == :admin,
                                select: status

selector = ClientStatus.where :user_id == id and :status == :admin,
                                select: :status
# etc.

Thanks,
Michael

Unable to specify storage options,

I require ets: compressed for a project, but am unable to specify mnesia storage_options using just amnesia to define and create tables. I sent a pull request that has my intermediate fix for passing in ets and dets storage options during table definition and table creation.

Aborting a transaction?

Hi.

When inside a block like the following, how do I abort the ongoing transaction?

Amnesia.transaction do < reads, writes > end

When I called :mnesia.abort/1, the whole process died due to an uncaught exception. Same happened when I introduced some kind of matching error in a transaction.

How does it work? How can I abort a transaction in Amnesia?

New release?

In order to avoid having a dependency in master it would be nice to be able to anchor it to a release. Thanks! 😄

Created schema not recognised

Hi,

So, I use the install task to create a schema, but when I try to access the database, I get

** (exit) {:aborted, {:no_exists, [UserStore.User, #PID<0.395.0>]}}
     stacktrace:
       mnesia.erl:310: :mnesia.abort/1
       lib/amnesia/table.ex:484: Amnesia.Table.read!/2
       (app) lib/app/connection_store.ex:6: App.UserStore.User.read!/1
       (app) lib/app/connection_store.ex:73: App.ConnectionStore.lookup/1
       test/app_test.exs:6

My Schema is created with:

deftable User, 
    [:pid,:username,:bytes,:datetime], 
    type: :set, 
    index: [:username] do
    @type t :: %User{pid: pid, 
                           username: String.t, 
                           bytes: non_neg_integer, 
                           datetime: {{non_neg_integer, non_neg_integer, non_neg_integer}, {non_neg_integer, non_neg_integer, non_neg_integer}}}

On install, the necessary folders do get created.

foldl-like function that returns a struct

As far as I understand, if we want to run a function on each record on table (to filter, or map...), we could use the foldl/3 function.

But, it's hard to use, because the value passed to function isn't a struct, for example, if we have

defdatabase Database do
  deftable Account, [{:id, autoincrement}, :email, :account_number :amount],
  ...

and use

Amnesia.transaction do
  Database.Account.foldl([], &([&1 | &2]))
end

The return is something like [{Database.Account, 1, "[email protected]", "1", 100000, true}, {Database.Account, 2, "[email protected]", "2", 100000, true}].

A possible better return is [%Database.Account{account_number: "1", amount: 100000, email: "[email protected]", id: 1}, %Database.Account{account_number: "2", amount: 100000, email: "[email protected]", id: 2}].

Currently, I'm using a workaround to return an array of structs:

Amnesia.transaction do
  Database.Account.keys
  |> Enum.map(&Database.Account.read!/1)
end

But, maybe exists a better solution, and I think that it's useful to add on Amnesia.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.