elixir-mongo / mongodb Goto Github PK
View Code? Open in Web Editor NEWMongoDB driver for Elixir
License: Apache License 2.0
MongoDB driver for Elixir
License: Apache License 2.0
This is for the emj-dbconn branch
I believe my recent PR (#74) broke something. When trying to do simple queries, I am getting a protocol error.
However, if I execute the commands fast enough (copy / paste into iex
) the query works properly.
Let me know if there is anything else I can do to debug this.
12:44:41 (emj-dbconn) mongodb$ iex -S mix
Erlang/OTP 19 [erts-8.0] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false]
Interactive Elixir (1.3.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> {:ok, pid} = Mongo.start_link(database: "some_database")
{:ok, #PID<0.154.0>}
iex(2)> c = Mongo.find(pid, "users", %{})
%Mongo.Cursor{coll: "users", conn: #PID<0.154.0>, opts: [], query: %{},
select: nil}
iex(3)> Enum.to_list(c)
** (Mongo.Error) tcp recv: invalid argument - :einval
12:45:27.973 [info] Mongo.Protocol (#PID<0.154.0>) missed message: {:tcp, #Port<0.5043>, <<167, 25, 7, 0, 1, 245, 4, 0, 0, 0, 0, 0, 1, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 143, 1, 0, 0, 50, 24, 0, 0, 7, 95, 105, 100, 0, 82, 129, 95, ...>>}
12:45:27.973 [error] Mongo.Protocol (#PID<0.154.0>) disconnected: ** (Mongo.Error) tcp recv: invalid argument - :einval
(mongodb) lib/mongo/cursor.ex:40: anonymous fn/6 in Enumerable.Mongo.Cursor.start_fun/6
(elixir) lib/stream.ex:1121: anonymous fn/5 in Stream.resource/3
(elixir) lib/enum.ex:1626: Enum.reduce/3
(elixir) lib/enum.ex:2345: Enum.to_list/1
** (EXIT from #PID<0.152.0>) an exception was raised:
** (RuntimeError) disconnect/2 not implemented
(mongodb) lib/db_connection.ex:240: Mongo.Protocol.disconnect/2
(db_connection) lib/db_connection/connection.ex:178: DBConnection.Connection.disconnect/2
(connection) lib/connection.ex:767: Connection.disconnect/3
(stdlib) gen_server.erl:601: :gen_server.try_dispatch/4
(stdlib) gen_server.erl:667: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Interactive Elixir (1.3.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
12:45:27.978 [error] GenServer #PID<0.154.0> terminating
** (RuntimeError) disconnect/2 not implemented
(mongodb) lib/db_connection.ex:240: Mongo.Protocol.disconnect/2
(db_connection) lib/db_connection/connection.ex:178: DBConnection.Connection.disconnect/2
(connection) lib/connection.ex:767: Connection.disconnect/3
(stdlib) gen_server.erl:601: :gen_server.try_dispatch/4
(stdlib) gen_server.erl:667: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: {:"$gen_cast", {:disconnect, #Reference<0.0.6.118>, %Mongo.Error{code: nil, message: "tcp recv: invalid argument - :einval"}, %{database: "some_database", request_id: 0, socket: #Port<0.5043>, timeout: 5000, wire_version: 4, write_concern: %{w: 1}}}}
State: {Mongo.Protocol, %{database: "some_database", request_id: 0, socket: #Port<0.5043>, timeout: 5000, wire_version: 4, write_concern: %{w: 1}}}
When performing a sum aggregation and excluding _id from the final output I get an exception. The same operation can be completed from Mongo Client with out issue.
MongoDB shell version: 2.6.10
Mongo Client
db.www_test_com.aggregate(
[
{ $group: { _id: '', total: { $sum: "$bytes_sent" } } },
{ $project: { _id: 0, total: '$total' } }
]
);
{ "total" : 85000 }
Using Mongo.aggregate with project
resp = Mongo.aggregate(MongoPool, "www_test_com",
[
%{'$group': %{ '_id': false, 'total': %{ '$sum': "$bytes_sent" }}},
%{'$project': %{ '_id': 0, 'total': '$total' }}
]
) |> Enum.to_list
** (EXIT) an exception was raised:
** (CaseClauseError) no case clause matching: {:ok, %Mongo.ReadResult{cursor_id: 0, docs: [%{"code" => 15992, "errmsg" => "exception: disallowed field type Array in object expression (at 'total')", "ok" => 0.0}], from: 0, num: 1}}
Using Mongo.aggregate without project
resp = Mongo.aggregate(MongoPool, "www_test_com",
[
%{'$group': %{ '_id': false, 'total': %{ '$sum': "$bytes_sent" }}}
]
) |> Enum.to_list
IO.inspect(resp)
[%{"_id" => false, "total" => 85000}]
When I try to do a basic search as per the examples, I get the above error.
Hi @ericmj! I have been recently working with massive datasets and more or less complex aggregate functions to retrieve data, operations that sometimes last 2-3 minutes. I've set the HTTP timeout high but even with this option if my process lasts more than five secs the GenServer.call raises an exception because it is always called with the default timeout for GenServer operations(5000ms), do you think is worth to send the same time as in HTTP timeout for mongo? For the moment I have this in my local project:
GenServer.call(conn, {:find, coll, query, select, opts}, 60_000)
60_000
is a hardcoded limit I put only to try, and it worked.
After I insert some data I get this:
%Mongo.InsertOneResult{inserted_id: #BSON.ObjectId<57dc602dfa2d310e03f95be8>}
That's great, but I have no idea what to do with this BSON.Objectid
I would like to get back maybe a map of all the data I just inserted or the Mongo ID so I can look up the record I just inserted.
Can anyone help me figure out how to do this?
Would like the ability to also define the database when defining the pool definition itself, instead of specifying the database on start_link. Since the pool is directly tied to the database, this keeps it cleaner.
The existing approach is:
defmodule DatabasePool do
use Mongo.Pool, name: MODULE, adapter: Mongo.Pool.Poolboy
end
but then elsewhere having to perform a DatabasePool.start_link(database: "my_database").
Would also like the ability to do this instead...
defmodule DatabasePool do
use Mongo.Pool, name: MODULE, adapter: Mongo.Pool.Poolboy, database: "my_database"
end
DatabasePool.start_link
You can treat a cursor as an Elixir Stream, but this should really be documented. I'll make a PR for this soon.
Would be useful to encode to JSON, e.g. map of strings:
BSON.encode_json(doc)
%{"_id" => "559c3c676b92c9bf5c0898fd", "name" => "Travis"}
I'm pretty new to Elixir and this is above my level still, but when I can do it I'll make a PR if there isn't already.
Hi,
I want to suggest you to move BSON lib to a separate HEX package, since your implementation works much (more than 50%) faster and predictable than bson
package that have 10k downloads right now.
At the moment I'm used to copy-paste your code and make some cleanup, since making DB driver as dependency to parse BSON would be an ugly solution.
I see this one is started about a year ago while checkiz/elixir-mongo is a lot older. What should one chose when starting fresh and for what reason? Thanks for your feedback :-)
distinct/find/insert_one/insert_many/delete_one/delete_many/replace_one/update_one/update_many/save_one/save_many including MongoCursors should support :database option to specify a database to talk with.
Hi, on one of my collections I get the following error:
** (FunctionClauseError) no function clause matching in BSON.Decoder.type/2
(mongodb) lib/bson/decoder.ex:10: BSON.Decoder.type(1, <<1, 0, 0, 0, 0, 0, 240, 127, 1, 49, 0, 1, 0, 0, 0, 0, 0, 240, 127>>)
lib/bson/decoder.ex:10: BSON.Decoder.type(1, <<0, 0, 0, 0, 0, 0, 240, 127, 1, 112, 114, 111, 106, 101, 99, 116, 101, 100, 69, 84, 99, 70, 114, 111, 109, 104, 105, 115, 116, 111, 114, 121, 0, 170, 241, 210, 77, 98, 16, 201, 63, 2, 67, 73, 77, 73, 83, 83, 116, 97, ...>>)
I am pulling from master since it seems a similar issue was resolved recently, but it still breaks. I'm connecting to an existing database so I need to be able to access the values. Is there a workaround for this type of issue before it is fixed?
I think it is caused by a 2 element array that has nan
s with typedouble
.
Thanks.
hi, is there a way to make BSON.ObjectId.decode!/1 available?, I'm passing a string as parameter
15:01:10.138 [error] #PID<0.287.0> running Estorage.Router terminated
Server: localhost:4001 (http)
Request: POST /uploadfile
** (exit) an exception was raised:
** (UndefinedFunctionError) function BSON.ObjectId.decode!/1 is undefined or private
(mongodb) BSON.ObjectId.decode!("579a2daf189fe3491028a3c6")
(estorage) lib/estorage/users.ex:7: Estorage.Users.getById/1
(estorage) lib/estorage/manager.ex:8: Estorage.Manager.upload_file/5
(estorage) lib/estorage/router.ex:18: anonymous fn/1 in Estorage.Router.do_match/4
(estorage) lib/estorage/router.ex:2: Estorage.Router.plug_builder_call/2
(plug) lib/plug/adapters/cowboy/handler.ex:15: Plug.Adapters.Cowboy.Handler.upgrade/4
(cowboy) src/cowboy_protocol.erl:442: :cowboy_protocol.execute/4
Hi @ericmj. Thanks for this. I was wondering if you would be interested in a PR to support authentication via an Authentication Database? If so, would master be the correct branch to submit the PR to? I've noticed some tests are failing on master and that there is another branch that seems to be active. Thanks :)
I get an error when the BSON decoder tries to decode the machine identifier bytes of an ObjectId.
** (MatchError) no match of right hand side value: <<111, 230, 80, 74, 154, 74, 223, 8>>
stacktrace:
(mongodb) lib/bson/decoder.ex:98: BSON.Decoder.document/1
(mongodb) lib/bson/decoder.ex:6: BSON.Decoder.decode/1
test/controllers/user_controller_test.exs:27
The original binary was <<86, 206, 224, 104, 111, 230, 80, 74, 154, 74, 223, 8>>
The code below would throw a duplicate ID error, but the rescue doesn't work.
What's the right way to catch/rescue an operation so that I can log the message?
try do
Mongo.insert_one(MongoPool, "articles",%{"_id" => "aaa"})
Mongo.insert_one(MongoPool, "articles",%{"_id" => "aaa"})
rescue
e in RuntimeError -> IO.inspect e
end
If mongo is inaccessible (offline), an attempt to use any Mongo functions results in an exception.
The Connection is returning a {:error, :closed} upstream, but the local function handler tries to raise all errors of the pattern {:error, e} causing the exception.
The Connection error should be converted to some kind of %Mongo.Error{} instead, so the raise function consistently reports these exceptions.
Hi,
I'd like to query the connected database to get a list of collections?
Can you give an example of how to do that?
Thanks
-John
@ericmj I think that adding replica set support will really boost the elixir / phoenix community.
Using mongodb in production without replica set it's not really possible. My company has a project that I really would like to migrate to phoenix and it's impossible without replica set support. I think there is a lot of developers in the same position. If you need help I can try and add the replica set support myself. If you already working on it I would like to help anyway I can.
I'm sure this is due to my inexperience, but I'm trying to find an example of setting up a stream from a cursor.
I can see a number flaws in my approach and figure there is likely a better way to do this.
def doc_stream(some_criteria) do
Stream.resource(
fn -> iteration end,
fn(iteration) ->
IO.puts "iteration: #{iteration}"
data = Mongo.find(SomePool, "collection_name", %{"field" => some_criteria, limit: 1, skip: iteration)
if Enum.empty?(data) do
{:halt, iteration}
else
{[Enum.at(data, 0)], iteration + 1}
end
end,
fn(iteration) -> iteration end
)
end
doc_stream |> Enum.take(10)
My goal is to not put a ton of pressure on the mongo servers when trying to work through massive collections in batches. I'd like to utilize streams and async processes to work on small chunks and only pull the next n-number of documents when its time to use them.
My issue with what I have is that if I take 10, its going to fire 10 queries at the database I assume. That is probably not ideal.
But before I sink more time into this and reinvent the wheel, I figured one of you may have a better approach or maybe this is silly altogether?
Please, add a LICENSE file for this project.
BTW, congrats! ๐
BSON encoding/decoding of values does not pay attention to the endianness of the storage when doing binary pattern matching. All types which have a possible endianness should be serialized to little endian per the BSON spec.
Elixir (and Erlang) doesn't natively support those special floating point values so if we see any such value from Mongo, we'll simply throw up an error.
** (FunctionClauseError) no function clause matching in BSON.Decoder.type/2
(mongodb) lib/bson/decoder.ex:10: BSON.Decoder.type(1, <<0, 0, 0, 0, 0, 0, 248, 127>>)
It might be possible to binary match to detect those values and return our own special atoms as discussed here: https://groups.google.com/forum/#!topic/capnproto/T6jEFu38rtI
and here: kaos/ecapnp@a4e3221#diff-d28a6e8472db5dc7149ae3347d12566aR84
If you are just recursion processing results, and pass a blank result set, it crashes on this.
** (FunctionClauseError) no function clause matching in Mongo.many_docs/1
(mongodb) lib/mongo.ex:449: Mongo.many_docs([])
(mongodb) lib/mongo.ex:130: Mongo.insert_many/4
(twitch_discovery) lib/index/stream.ex:2: TwitchDiscovery.Index.Stream.mongo_save_many/1
(twitch_discovery) lib/index/stream.ex:2: TwitchDiscovery.Index.Stream.get_next/1
(twitch_discovery) lib/index.ex:41: TwitchDiscovery.Index.index/0
Hi there. Thanks for this great library! I have been struggling with a few other libraries that are basically dead. Everything works as expected until this issue came up. I can't figure out how to get past it, and using the code for find
that you're using in the tests. I understand this is basically pre-alpha, but my requirements are limited at the moment.
defmodule MongoPool do
use Mongo.Pool, name: __MODULE__, adapter: Mongo.Pool.Poolboy
end
collection = "broadcasts0"
Mongo.find(MongoPool, collection, %{} |> Enum.to_list
** (exit) an exception was raised:
** (Protocol.UndefinedError) protocol Enumerable not implemented for %Mongo.Cursor{coll: "broadcasts-0", opts: [], pool: MongoPool, query: %{}, select: nil}
(elixir) lib/enum.ex:1: Enumerable.impl_for!/1
(elixir) lib/enum.ex:112: Enumerable.reduce/3
(elixir) lib/enum.ex:1265: Enum.reduce/3
(elixir) lib/enum.ex:1807: Enum.to_list/1
Thanks ๐
Say I have an incremental id, in elixir i set "_id"=> "incrId()", "timest"=>"db.eval("Date()")"
Mongo.insert_one(MongoPool, "test", %{
"_id"=> "incrId()",
"timest"=> "db.eval(\"Date()\")"
})
This makes the record saved without evaluating the function.Neither way works. Using the commandline for mongo this behavior does not happen as the functions get evaluated as they are inserted.
Was writing some more documentation and noticed naming of "remove" from Mongo is named as "delete_*". The main Mongo API uses remove()
, so I was looking for that in the code. Connection is using remove though, so not sure why its like this. Could just be something that's going to change later.
Summary: Why is Mongo
using delete_*
instead of remove
?
If I run a Mongo.find query on a large mongo collection in a busy server with:
cursor = Mongo.find(MongoPool, "userdata", %{state: "ready"}, batch_size: 1000)
After a few hours of running I always get a timeout into :get_more:
** (exit) exited in: GenServer.call(#PID<0.169.0>, {:get_more, "userdata", 17595059170, [batch_size: 1000]}, 5000)
** (EXIT) time out
(elixir) lib/gen_server.ex:564: GenServer.call/3
(stdlib) timer.erl:181: :timer.tc/2
lib/mongo/pool/poolboy.ex:36: Mongo.Pool.Poolboy.run/2
lib/mongo/pool.ex:142: Mongo.Pool.run_with_log/5
lib/mongo/cursor.ex:65: anonymous fn/3 in Enumerable.Mongo.Cursor.next_fun/2
(elixir) lib/stream.ex:1099: Stream.do_resource/5
(elixir) lib/enum.ex:1477: Enum.reduce/3
(elixir) lib/enum.ex:1092: Enum.map/2
The :timeout connection parameter is not used for :get_more so the only way to fix it is to manually edit this line: https://github.com/ericmj/mongodb/blob/943f6bcefff916e2e8b572fad1c9d79d55235490/lib/mongo/connection.ex#L64
and change it to:
GenServer.call(conn, {:get_more, coll, cursor_id, opts}, 50000)
With 50000 everything works fine and I no longer get any timeouts. So I think the :timeout connection parameter should also be use here or maybe create a new parameter.
Hi there, I'm an elixir noob trying to insert a date into mongo, but I can't figure out to to create the BSON.DateTime object properly. I'm currently doing this:
utc = :calendar.datetime_to_gregorian_seconds(:calendar.universal_time)
%BSON.DateTime{utc: get_utc_timestamp}
When it saves to mongo, it shows up as 1972-01-07...
.
What's the correct way to create a DateTime for now
?
Is pool still available? Can't seem to use it with v2.1.. And if it's not used anymore, what is the alternative?
Sort is made with atoms, and can't do this with them. I think they should remain as a string keys.
Mongo.find(MongoPool, %{}, [sort: [game.name: -1]])
Thanks ๐
I saw a previous issue that was the same as mine but the solution for them did not work for me (adding a name parameter to start_link). This is my first time using the library (version 2.0.0) and I followed the instructions in the README. I've got a MongoPool module:
defmodule MongoPool do
use Mongo.Pool, name: __MODULE__, adapter: Mongo.Pool.Poolboy
end
and when the app tries to compile I get the following error:
** (CompileError) lib/intake_zen/mongo_pool.ex:2: module Mongo.Pool is not loaded and could not be found
(elixir) expanding macro: Kernel.use/2
lib/intake_zen/mongo_pool.ex:2: MongoPool (module)
(elixir) lib/kernel/parallel_compiler.ex:116: anonymous fn/4 in Kernel.ParallelCompiler.spawn_compilers/1
I searched through the driver repo and can't find Mongo.Pool anywhere. Is this module no longer available?
I'm sure I may be missing something here, but after an insert I get back a BSON.ObjectId
like #BSON.ObjectId<1e0b239999a1014ed1b35a84>
.
What is the easiest way to get a string representation of the actual _id
for later use?
I've tried something like %BSON.ObjectId{value: object_id} = #BSON.ObjectId<1e0b239999a1014ed1b35a84>
, but then I have a binary.
Feels like I'm missing something real simple here. Thanks!
When I follow the instructions on the Readme I get an error doing an insert:
iex(4)> Mongo.insert_one(MongoPool, "crawl_report", %{"field" => 10})
** (ArgumentError) argument error
(stdlib) :ets.lookup_element(Mongo.IdServer, :machineprocid, 2)
lib/mongo/id_server.ex:52: Mongo.IdServer.new/0
lib/mongo/connection.ex:134: Mongo.Connection.add_id/1
lib/mongo/connection.ex:102: Mongo.Connection.assign_ids/1
lib/mongo/connection.ex:79: Mongo.Connection.insert/4
(stdlib) timer.erl:181: :timer.tc/2
lib/mongo/pool/poolboy.ex:36: Mongo.Pool.Poolboy.run/2
lib/mongo/pool.ex:142: Mongo.Pool.run_with_log/5
However if I manually run Mongo.IdServer.start_link the insert works:
iex(3)> Mongo.insert_one(MongoPool, "crawl_report", %{"field" => 10})
** (ArgumentError) argument error
(stdlib) :ets.lookup_element(Mongo.IdServer, :machineprocid, 2)
lib/mongo/id_server.ex:52: Mongo.IdServer.new/0
lib/mongo/connection.ex:134: Mongo.Connection.add_id/1
lib/mongo/connection.ex:102: Mongo.Connection.assign_ids/1
lib/mongo/connection.ex:79: Mongo.Connection.insert/4
(stdlib) timer.erl:181: :timer.tc/2
lib/mongo/pool/poolboy.ex:36: Mongo.Pool.Poolboy.run/2
lib/mongo/pool.ex:142: Mongo.Pool.run_with_log/5
iex(3)> Mongo.IdServer.start_link
{:ok, #PID<0.203.0>}
iex(4)> Mongo.insert_one(MongoPool, "crawl_report", %{"field" => 10})
{:ok,
%Mongo.InsertOneResult{inserted_id: #BSON.ObjectId<1dfbc07783152906278464a1>}}
Is this expected behavior or is something wrong?
This would be pretty cool. They also have a nice guide for it.
How can I query for document based on ObjectID as string? The only way I could get it to work is
to use Mongo.Ecto.ObjectID.dump from Mongo.Ecto project, like this:
Mongo.find(MongoPool, "some_collection", %{_id: elem(Mongo.Ecto.ObjectID.dump("55e724bf44f4173c876919f9"),1) })
is there a way to do it and not depend on an external project?
Currently I'm migrating a lib which using elixir-mongo as dependency which it doesn't require a database when creating a connection(which should be much more nature as well). The reason why I'm doing so is because elixir-mongo did purge all values's key into atom which might cause a huge memory leakage if it is running in a long term,and looks like their project is unmaintained for a while as well.
Everything looks fine for mongoldb as well, but I can't find anyway to create a clean connection to the database both with Mongo.Connection or MongoPool, is there any way I can circumvent with this restricts.
And also I did notice all find/update or db actions are taking collection as the first argument, so from my understanding this is a designed pattern, but indeed I need to maintain a single connection and talks to multiple db in the same time, is there anyway I can use it except for fork a new version?
Hi!
I'm trying to use the package in a project that I'm working on, but if I try to use DBConnection.Poolboy
as specified in the instructions, I get the following error:
iex(11)> Mongo.find(:mongo, "products", %{}, limit: 20, pool: DBConnection.Poolboy) |> Enum.to_list
** (UndefinedFunctionError) function :invalid_message.exception/1 is undefined (module :invalid_message is not available)
:invalid_message.exception([])
(db_connection) lib/db_connection.ex:925: DBConnection.checkout/2
(db_connection) lib/db_connection.ex:741: DBConnection.run/3
(db_connection) lib/db_connection.ex:1132: DBConnection.run_meter/3
(db_connection) lib/db_connection.ex:636: DBConnection.execute/4
(mongodb) lib/mongo.ex:386: Mongo.kill_cursors/3
(elixir) lib/stream.ex:1129: Stream.do_resource/5
(elixir) lib/enum.ex:1627: Enum.reduce/3
(elixir) lib/enum.ex:2346: Enum.to_list/1
These are my deps:
* db_connection 1.1.0 (Hex package) (mix)
locked at 1.1.0 (db_connection) b2b88db6
ok
* mongodb 0.2.0 (Hex package) (mix)
locked at 0.2.0 (mongodb) 95f85074
ok
* poolboy 1.5.1 (Hex package) (rebar)
locked at 1.5.1 (poolboy) 6b461639
ok
And this is how I have added the Mongo
worker to the main supervision tree:
defp children do
[
...
worker(Mongo, [mongo_config]),
...
]
end
defp mongo_config do
[
name: :mongo,
pool: DBConnection.Poolboy,
username: System.get_env("MONGODB_USERNAME"),
password: System.get_env("MONGODB_PASSWORD"),
database: System.get_env("MONGODB_DBNAME"),
]
end
The thing is that if I remove the pool from the config, and run the same query without the pool, it actually works. What might be going wrong?
Thanks in advance!
Hello, I'm trying to connect a MongoDB version 3.2 but it seems it isn't using SCRAM authentication, how can I specify that in my connection options?
{:ok, _} = MongoPool.start_link(
hostname: server,
username: username,
password: password,
database: dbname
)
** (Mix) Could not start application ejobs: exited in: JobsMonitor.Start.start(:normal, [])
** (EXIT) exited in: GenServer.call(#PID<0.149.0>, {:find, "bz_administrators", %{}, nil, [batch_size: 1000]}, 5000)
** (EXIT) %Mongo.Error{code: 18, message: "auth failed for 'admin': Authentication failed."}
Values passed as connection parameters are ok
I want to use this on Heroku where I am provided a full connection string as an environment variable. Is there any way to connect with a string in the following format?
mongodb://heroku_****:****@***.mlab.com:17545,***.mlab.com:17545/heroku_*****?replicaSet=rs-****
iex(45)> Mongo.find(MongoPool, "tigerkakor", %{}, skip: 10, cursor_timeout: false)
%Mongo.Cursor{coll: "tigerkakor", opts: [], pool: MongoPool, query: %{},
select: nil}
It seems there are intermittent failures in the test/mongo/connection_test.exs file, for the "auth wrong" scenario on line 31.
As an example travis run:
https://travis-ci.org/ericmj/mongodb/jobs/141147103
Is it possible this is the kind of message ordering issue that should be handled using special process tests, as described here:
http://blog.carbonfive.com/2016/06/28/special-processes-in-otp/ ?
Take a look at elrmongo
So, if you create a tailable cursor, you can use the cursor as a Stream; however, the functionality that a tailable cursor adds (allowing you to see newly inserted documents for a cursor as they are added) doesn't really support streams. We'd need some way to have a supervised process that sends new results to another process as they are available. Here's the code for how RethinkDB handles this:
Add a .drop() function to drop a collection.
We performed it manually using a run_command but if the collection does not exist, an uncaught exception occurs...
** (CaseClauseError) no case clause matching: %{"errmsg" => "ns not found", "ok" => 0.0}
As a guidepost, within our wrapper lib, we're handling like this presently..
def drop_collection(collection) do
try do
%{"ok" => 1.0} = Mongo.run_command(@pool, %{:drop => collection})
:ok
rescue
e in CaseClauseError ->
Logger.debug "#{inspect e}"
:error
end
end
I'm getting an error running queries against the oplog on a MongoDB replica set. MongoDB v 2.6.5 / Elixir 1.0.5.
An iex session that illustrates this:
-> % iex -S mix
Erlang/OTP 18 [erts-7.0.2] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Interactive Elixir (1.0.5) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> defmodule MongoPool do use Mongo.Pool, name: __MODULE__, adapter: Mongo.Pool.Poolboy end {:module, MongoPool,
<<70, 79, 82, 49, 0, 0, 9, 80, 66, 69, 65, 77, 69, 120, 68, 99, 0, 0, 1, 111, 131, 104, 2, 100, 0, 14, 101, 108, 105, 120, 105, 114, 95, 100, 111, 99, 115, 95, 118, 49, 108, 0, 0, 0, 2, 104, 2, ...>>,
:ok}
iex(2)> {:ok, _} = MongoPool.start_link(database: "local")
{:ok, #PID<0.134.0>}
iex(3)> Mongo.find(MongoPool, "oplog.rs", %{}) |> Enum.take(1)
** (exit) exited in: GenServer.call(#PID<0.150.0>, {:find, "oplog.rs", %{}, nil, [batch_size: 1000]}, 5000)
** (EXIT) an exception was raised:
** (FunctionClauseError) no function clause matching in Mongo.Connection.handle_info/2
lib/mongo/connection.ex:346: Mongo.Connection.handle_info({:tcp, #Port<0.5509>, <<0, 3, 0, 0, 0, 82, 85, 0, 2, 49, 56, 48, 0, 3, 0, 0, 0, 82, 87, 0, 2, 49, 56, 49, 0, 3, 0, 0, 0, 82, 69, 0, 2, 49, 56, 50, 0, 3, 0, 0, 0, 66, 76, 0, 2, 49, 56, 51, ...>>}, {:ok, %{auth: [], database: "local", opts: [backoff: 1000, port: 27017, hostname: 'localhost'], queue: %{0 => %{command: :find, from: {#PID<0.127.0>, #Reference<0.0.3.911>}, params: nil}}, request_id: 1, socket: #Port<0.5509>, tail: <<84, 120, 47, 0, 85, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 8, 0, 0, 0, 138, 133, 8, 97, 208, 0, 0, 0, 0, 0, 0, 0, 232, 3, 0, 0, 242, 11, 0, 0, 17, 116, ...>>, timeout: 5000, wire_version: 2, write_concern: [w: 1]}})
lib/connection.ex:798: Connection.handle_async/3
(stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
(stdlib) gen_server.erl:681: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:239: :proc_lib.init_p_do_apply/3
18:55:47.367 [error] GenServer #PID<0.150.0> terminating
Last message: {:tcp, #Port<0.5509>, <<0, 3, 0, 0, 0, 82, 85, 0, 2, 49, 56, 48, 0, 3, 0, 0, 0, 82, 87, 0, 2, 49, 56, 49, 0, 3, 0, 0, 0, 82, 69, 0, 2, 49, 56, 50, 0, 3, 0, 0, 0, 66, 76, 0, 2, 49, 56, 51, ...>>}
State: {:ok, %{auth: [], database: "local", opts: [backoff: 1000, port: 27017, hostname: 'localhost'], queue: %{0 => %{command: :find, from: {#PID<0.127.0>, #Reference<0.0.3.911>}, params: nil}}, request_id: 1, socket: #Port<0.5509>, tail: <<84, 120, 47, 0, 85, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 8, 0, 0, 0, 138, 133, 8, 97, 208, 0, 0, 0, 0, 0, 0, 0, 232, 3, 0, 0, 242, 11, 0, 0, 17, 116, ...>>, timeout: 5000, wire_version: 2, write_concern: [w: 1]}}
** (exit) an exception was raised:
** (FunctionClauseError) no function clause matching in Mongo.Connection.handle_info/2
lib/mongo/connection.ex:346: Mongo.Connection.handle_info({:tcp, #Port<0.5509>, <<0, 3, 0, 0, 0, 82, 85, 0, 2, 49, 56, 48, 0, 3, 0, 0, 0, 82, 87, 0, 2, 49, 56, 49, 0, 3, 0, 0, 0, 82, 69, 0, 2, 49, 56, 50, 0, 3, 0, 0, 0, 66, 76, 0, 2, 49, 56, 51, ...>>}, {:ok, %{auth: [], database: "local", opts: [backoff: 1000, port: 27017, hostname: 'localhost'], queue: %{0 => %{command: :find, from: {#PID<0.127.0>, #Reference<0.0.3.911>}, params: nil}}, request_id: 1, socket: #Port<0.5509>, tail: <<84, 120, 47, 0, 85, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 8, 0, 0, 0, 138, 133, 8, 97, 208, 0, 0, 0, 0, 0, 0, 0, 232, 3, 0, 0, 242, 11, 0, 0, 17, 116, ...>>, timeout: 5000, wire_version: 2, write_concern: [w: 1]}})
lib/connection.ex:798: Connection.handle_async/3
(stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
(stdlib) gen_server.erl:681: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:239: :proc_lib.init_p_do_apply/3
(elixir) lib/gen_server.ex:356: GenServer.call/3
(stdlib) timer.erl:181: :timer.tc/2
lib/mongo/pool/poolboy.ex:34: Mongo.Pool.Poolboy.run/2
lib/mongo/pool.ex:140: Mongo.Pool.run_with_log/5
lib/mongo/cursor.ex:33: anonymous fn/6 in Enumerable.Mongo.Cursor.start_fun/6
(elixir) lib/stream.ex:1014: anonymous fn/5 in Stream.resource/3
(elixir) lib/enum.ex:1740: Enum.take/2
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.