grahamjenson / hapiger Goto Github PK

Hello, I am running RethinkDB on my Mac with rethinkdash driver installed and I am evaluating Hapiger performance with rethinkdb as esm.
Write throughput is about 150 events per second. I have 4 different actions.
While writing it is almost impossible to execute recommendation requests - times out after 3min.
When I stop writing with 220K events insterted, recommendation request completes in 30-40s which is very slow. I am looking for <100ms response time.
Anything I can change improve in default config?
Thank you.

I just wanted to know does expired recommendations gets automatically cleaned up form the database. In my use case, recommendations gets fed based on user activity. So I'm expecting recommendations will be fed in massive amounts. I have also set a two week expiry for recommendations.

My concern is the presence of expired/redundant recommendations in the db. I have validated that expired recommendations does not affect the present recommendations but simply the presence of data in the db may affect the performance.

Has removal of redundant data been automated or should I do it from my part?

I follow the quick start guide, and running hapiger with in-memory event store gives a WARN:
(node:19148) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.

Anyways, the server starts up, but after feeding it with the events of Bob and Alice, it never returns any recommendation:

curl -X POST 'http://localhost:3456/recommendations' -d '{
    "namespace": "movies",
    "person": "Alice",
    "configuration": {
      "actions" : {"view": 5, "buy": 10}
    }
}'

returns:
{"recommendations":[],"neighbourhood":{"Alice":1},"confidence":0}

RethinkDB is an abandonned project

https://rethinkdb.com/blog/rethinkdb-shutdown/

Hello, I'm trying your recommendation system HapiGER. I'm using RethinkDB. I'm tracking actions, persons and thinks from my website and sending them to HapiGER. That's working perfect, I can see all the data correctly stored in RethinkDB but when I ask HapiGER for a recommendation for a person with some history on the system (or none, it's the same effect), HapiGER doesn't work because it encounter an error with ReqlDriver and it can't connect to RethinkDB in order to do the query.
This is the console extract from my server (Ubuntu 12.04.05 LTS):

$ hapiger -v --es rethinkdb --esoptions '{"host":"devserver","port":28015,"db":"test"}' -n 10 -v
{ esm: 'rethinkdb',
esmoptions: { host: 'devserver', port: 28015, db: 'test' },
port: 3456,
minimum_history_limit: 1,
similar_people_limit: 25,
related_things_limit: 10,
recommendations_limit: '10',
recent_event_days: 14,
previous_actions_filter: [],
compact_database_person_action_limit: 1500,
compact_database_thing_action_limit: 1500,
person_history_limit: 500,
crowd_weight: 0 }
Creating a pool connected to devserver:28015
Starting Server on 3456
Unhandled Error { [ReqlDriverError: Unrecognized option useOutdated in run. Available options are readMode , durability , noreply , timeFormat , groupFormat: , profile , binaryFormat , cursor , stream .]
message: 'Unrecognized option useOutdated in run. Available options are readMode , durability , noreply , timeFormat , groupFormat: , profile , binaryFormat , cursor , stream .' } ReqlDriverError: Unrecognized option useOutdated in run. Available options are readMode , durability , noreply , timeFormat , groupFormat: , profile , binaryFormat , cursor , stream .
at [object Object]._onTimeout (/usr/lib/node_modules/hapiger/node_modules/ger/node_modules/rethinkdbdash/lib/term.js:186:44)
at Timer.listOnTimeout as ontimeout

150928/061010.767, [response], http://devserver:3456: get /default/recommendations {"person":"4de204f11fe378b3","action":"view"} 500 (47ms)

I think it could be an error for incorrect compatibility between Driver and RethinkDB versions. I'm really interested in your work, GER system it's amazing and powerful, great work Graham.

Best regards, Sofia.

will be nice if cors will work out of the box or at least with some command line flag

Wondering if there is a way to limit the recommendations returned? In my case i want to continuously return a set of 10 or 20. Once the database grows i feel like returning 10,000 recommendations isn't going to be beneficial and it's definitely going to slow things down.

Project has no coverage report, I would like to introduce this in order to understand and increase coverage and tests on the module.

I will also:

1. bump mocha and created npm-shrinkwrap.json in order to avoid having to fight with dependencies when something gets updated and breaks the build.

2. Update readme adding RethinkDB ESM and updating changelog

Hi, I just installed HapiGER, but it won't work, but there is also no error.

My setup

Installed HapiGER:

npm install -g hapiger

Created a database in my Postgres (hapiger) and start HapiGER as explained in your documentation:

hapiger --es pg --esoptions '{
    "connection":"postgres://localhost/hapiger"
  }'

It seems to be connecting, my database gets connected to and without having the database HapiGER will not start due to an error ("Knex:Error Pool2 - error: database "xxx" does not exist")

My issue

But now, when creating a namespace, e.g:

curl -X POST 'http://localhost:3456/namespaces' -d'{
    "namespace": "movies"
  }'

--> Returns {"namespace":"movies"}%

and and event:

curl -X POST 'http://localhost:3456/events' -d '{
    "events": [
    {
      "namespace": "movies",
      "person":    "Alice",
      "action":    "view",
      "thing":     "Harry Potter"
    }
  ]
}'

--> Returns {"events":[{"person":"Alice","action":"view","thing":"Harry Potter","created_at":"2018-01-19T12:08:53.873Z"}]}%

...nothing is stored in the database. Or retrievable at all..

Any clues, or what to look out for?

Hi,
First, i want to thanks to you for great plugin.Its really a good plugin and informative.
Problem:
Events:

curl -X POST 'http://localhost:3456/events' -d '{
    "events": [
    {
      "namespace":  "test",
      "person":     "[email protected]",
      "action":     "view",
      "thing":      "IPOD"
    }
  ]
}'

curl -X POST 'http://localhost:3456/events' -d '{
    "events": [
    {
      "namespace":  "test",
      "person":     "[email protected]",
      "action":     "view",
      "thing":      "IPOD"
    }
  ]
}'

curl -X POST 'http://localhost:3456/events' -d '{
    "events": [
    {
      "namespace":  "test",
      "person":     "[email protected]",
      "action":     "buy",
      "thing":      "IPOD",
      "expires_at": "2016-09-10"
    }
  ]
}'`

recommendations:

`curl -X POST 'http://localhost:3456/recommendations' -d '{
    "namespace": "test",
    "person": "[email protected]",
    "configuration": {
      "actions" : {"view": 5, "buy": 10}
    }
}'

returns: IPOD

But when i raise event for '[email protected]'

curl -X POST 'http://localhost:3456/events' -d '{
    "events": [
    {
      "namespace":  "test",
      "person":     "[email protected]",
      "action":     "buy",
      "thing":      "IPOD",
      "expires_at": "2016-09-10"
    }
  ]
}'

and send request for recommendation it still returns "IPOD". Why? What should i do?

2: hapiger using postgres works in windows or not?

I'm doing a driver that maybe helpful for other people to be able to use hapiger with their nodejs apps
https://www.npmjs.com/package/hapigerjs

cheers

I was wondering if their is anything planned or done to solve the cold-start problem. The scenario is you haven't taken any actions so no recommendations are returned but since no recommendations are returned you can't take action on anything.

bam

Hi! Is there any way to exclude items from recommendations? Imagine an e-commerce app in which I want to specify that some items are no longer available, but I still want to use that data for similarity calculations. Thanks!

Started out with the default in-memory option, everything went fine as expected.
Tried recommended pgsql for persistence afterwards, very same data set, import went fine.

Same query as with in-memory didnt return anything.
So i gave mysql a shot, same behaviour, import fine, recommendation response is always empty.

any hints on what i might be missing?