hoodiehq / hoodie-server

:dog: Hapi plugin for Hoodie’s server core module

hoodie-server

Hapi plugin for Hoodie’s server core module

@hoodie/server integrates Hoodie’s server core modules:

• account-server
• store-server

Example

var Hapi = require('hapi')
var hoodie = require('@hoodie/server')
var PouchDB = require('pouchdb-core').plugin(require('pouchdb-mapreduce')).plugin(require('pouchdb-adapter-memory'))

var server = new Hapi.Server()
server.connection({
  host: 'localhost',
  port: 8000
})

server.register({
  register: hoodie,
  options: { // pass options here
    PouchDB: PouchDB,
    paths: {
      public: 'dist'
    }
  }
}, function (error) {
  if (error) {
    throw error
  }

  server.start(function (error) {
    if (error) {
      throw error
    }

    console.log(('Server running at:', server.info.uri))
  })
})

Usage

option	default	description
adminPassword	-	Password to login with admin account
paths.data	'.hoodie'	Data path
paths.public	'public'	Public path
PouchDB	–	PouchDB constructor. See also custom PouchDB builds.
account	{}	Hoodie Account Server options. account.admins are generated based on adminPassword option above. account.usersDb is hardcoded to _users at this point. account.secret is stored in hoodie-store/hoodie and will be generated if it does not yet exist.
store	{}	Hoodie Store Server options. store.couchdb, store.PouchDB are set based on the PouchDB option above. store.hooks.onPreAuth is set to bind user authentication for Hoodie Account to Hoodie Store.

Testing

Local setup

git clone https://github.com/hoodiehq/hoodie-server.git
cd hoodie-server
npm install

Run all tests

npm test

Contributing

Have a look at the Hoodie project's contribution guidelines. If you want to hang out you can join our Hoodie Community Chat.

License

Apache 2.0

hoodie-server's Issues

check for CouchDB version on startup

@caolan had a problem to get hoodie working, because the system CouchDB was used and it was outdated (0.11.x).

If hoodie requires a certain CouchDB version, we should check for that.

can I haz `hoodie new appname` and `hoodie start appname`?

[couch 1.3] api.* & couch.* ➔ db.*

For couch 1.3, we pretty much do not need the api. subdomain anymore.

I'd suggest to get rid of it and rename couch. to db.

what you think?

make hoodie compatible with *.nodejitsu.com domains

when deploying to nodejitsu, the app can be accessed at appname.jit.su, but not at appname.nodejitsu.com. But the latter seems to be the official way, at least it's used in the mails that nodejitsu is sending out (e.g. github deploy). And when I click on them, I get a 404

Set port explicitly

I dunno why, I guess some shid changed with cradle, but all of the sudden the port number defaulted to 5984, which caused the strange authentication error.

I'd suggest we do always set the port explicitly. So in the line return new worker(config); (hoodie-app.js), config.server should be http://couch.pocket.dev:80 instead of http://couch.pocket.dev.

Just to prevent problems in future.

set /modules/_security

when creating /modules database, do also create _security for it so nobody but the admin can access it.

deploy to heroku

as a 2nd option, besides nodejitsu

nodejitsu suppor for wildcard domains

just a note for later, it seems that there is nosupport for wildcard subdomains at nodejitsu:
https://twitter.com/jedschmidt/status/233199928309075968

we're fine with setting the admin / api / couch subdomains manually, but things like creating test users with the hoodie.admin module won't work, as it generates random hash subdomains to separate sessions

export internal state of plugin to hoodie-app, report to admin / logs

Add method to hoodie-worker that specific workers can implement to export state. for debugging

Use /_api and /_admin as default

Opening this ticket to discuss the effects this is likely to have on the various components that rely on separate origins. You can test it out on my branch of hoodie-app for now.

We're switching to this so we can provide a smooth experience on Linux and a consistent approach across platforms. I recommend still having the nice local-tld stuff for Mac so we have an app.dev domain, but sticking to app.dev/_api and app.dev/_admin so documentation and examples are cross-platform.

This currently causes some issues in hoodloader and probably pocket too (CC @gr2m @espy)

admin.appname.dev cannot be found

Run system self check on start

[setup] add _security to modules database

On initial setup, add _security to the modules database, so that only admins can access it.

Internal admin user

Currently we have a single couchdb _admin user for both workers and hoodie admin users. If the hoodie admin wants to change or reset password the workers will stop working (since they will not be notified of the new credentials). This change would also allow multiple hoodie admins to be added to a project (and they can act as the internal admin user through the admin.app.dev/_api endpoint).

1. Generate strong password for couchdb _admin user on install
2. prompt user to create a 'hoodie admin' account on install, give this account some role to indicate their elevated privileges.
3. pass the internally generated _admin credentials to workers
4. proxy any requests to the admin server's /_api so that requests are authenticated as the internal _admin user, provided the user has the appropriate role!

Admin passwords with special chars seem not to work on Nodejitsu

From the chat

[16:28:43] <masch>   [Thu 17:31] hi guys
[16:28:43] <masch>   [Thu 17:32] i just tryed to deploy to a free nodejitsu test account. Deploying worked good as far as i can see but when i try to start the app it doesn't work. I just goes back to "stopped" state every time
[16:28:43] <masch>   [Thu 17:33] ohh, jitsu logs works
[16:28:43] <masch>   [Thu 17:35] "reason: 'Name or password is incorrect.'," but HOODIE_ADMIN_PASS is set to the correct couchdb admin password
[16:28:43] <masch>   [Thu 17:35] theres a modules database created :/
[16:28:43] <masch>   [Thu 17:39] ahh nice. The password was just bad chosen. Took one out of [a-zA-Z0-9]+ now and that works.

[api] make a fallback path for browser that do not support CORS

IE < 10 does not support CORS.

I wonder we we could add a fallback, so that not only api.myapp.dev/_session, but also myapp.dev/api/_session would be redirected to the couch? I could build that into core hoodie quite easily as well.

And as soon as we have CouchDB 1.3, api.myapp.dev would be directly mapped while /api would still be handled by our revers proxy.

Thoughts?

give better error messages on setup

at the moment, I get the admin password prompt, although the couch could not even be reached. Add better error handling

multiple instances of the same app

Not tested yet. but thought I'd record this here while I'm thinking about it. We're storing passwords in npm config using app name as a key. If a user runs multiple instances of the same app (has the same name in package.json) with different couchdb passwords for each things will likely break.

hoodie-log.js crashes hoodie on first start

problem is, that the npm.load part gets executed before password is set:

npm.load(function(error, npm) {
  password = npm.config.get(name + "_admin_pass");
});

ECONNREFUSED on repeated npm start

here or in local-tld: repeat npm start will result in ECONNREFUSED reloading local-tld fixes it.

admin.appname.dev shows the app, not pocket

I've updated hoodie-app in my app, but still no luck.
Any idea? @caolan

[workaround] How to get hoodie work on linux

Until we make this proper, let's gather steps how to make hoodie work on linux

Hook up hoodiehq/pocket with hoodie-app-server

Windows!

After nearly a whole day in your IRC chat room me and mdlawson got hoodie working in windows (Yay! Everbody loves windows <3).

I wrote a guide for that: https://gist.github.com/Acconut/5383324 (Sorry for bad english).

I just wanted to let you know.

hoodie new appname --example hoodstrap

this is low prio. Just wanted to have a ticket for it.

I think it would be great to allow creating new hoodie apps based on several examples that are living at github.com/hoodiehq/example-*.git or something

add a 'bin' to package.json instead of hardcoding path

right now the package.json has:

    "start": "node node_modules/hoodie-app/lib/hoodie-app.js"

but if you had a bin field defined then npm is smart enough to resolve this to the correct path (would be same as above)

 "start": "hoodie-server"

this would also fix portability cause windows doesnt know what / means

make a load-testing / debug app

the app should provide a simple API for the following use cases

• create X new users
• create X objects (for user Y / each user)
• send X emails
• make X shares with Y objects

feel free to add more

Add port registry for non-local-tld environments

This will allows Linux users (and other platforms) to run multiple hoodie instances without having to worry about assigning port numbers. This has a more serious implication in the odd case the couch fails to be killed when hoodie-app dies hard and you might start a different hoodie-app and it'll think the previous couchdb instance is the new one (so all the data and dbs will be wrong!).

This means adding a .json config file somewhere, probably in ~/.hoodie/*

pass worker version as part of config

Besides the name, we'll need the version of the current worker soon. Can we find out the version of the workers and pass it along with its name?
https://github.com/hoodiehq/hoodie-app/blob/master/lib/hoodie-app.js#L125

Windows support

This issue is for discussion of windows support, now linux is supported.
To start off, here is my "todo list".

• hoodie-app/lib/config.js
  getConfig getCouch getHoodie all need windows stuff added.
• hoodie-app/lib/utils.js
  ensureDir mkdir -p isn't necessary on windows, and results in annoying litter
• node-multicouch needs this pull
• local-tld needs my rewrite
• hoodie-app/lib/localtld.js
  needs to use require("local-tld").config (with my rewrite)
• hoodie-cmd needs rewriting in node to support windows.

This is only what's obvious from a quickish scan through. After the node-multicouch and local-tld changes are made I can try setting everything up properly and test hoodie-app.

Switch to custom config store

Currently using NPM, we should have a config.json file for each app in the hoodie app directory where couchdb files are stored.

bail if run as `sudo`

[nodejitsu] cannot reach couch.appname.jit.su/

make magic hostname stuff optional

I think the autoconfiguring hostname stuff is cool but it is a leaky abstraction:

• makes deploying more complicated (e.g. recent nodejitsu bugs)
• hoodie server ignores requests from hostnames that it doesn't know about which makes custom domains hard cause you can't just point a domain at hoodie, you also have to tell hoodie about the domain
• makes cross platform support for the hoodie dev tools harder

it would be nice if I could "just" run hoodie server and have it accept requests from whatever use my own static file sever even

simplify hoodie.share module

little brother of https://github.com/hoodiehq/worker-shares/issues/10

satic server: fetch all only for non-asset urls

if url ends with one of the common extension like .js,.css,.png,.ico, ... the static server should not fallback to serve the root www/index.html. Instead, it should send a 404.

[api] Make GET /_all_dbs require an admin users

Is that possible at all? We definitely have to find a solution, it's a big security vulnerability

check out matchdep

Looks like an interesting option to load hoodie workers

https://github.com/tkellen/node-matchdep

request 2.0.0 changes `auth` parameter.

we need to depend on request 2.0.0 or switch from auth: "user:pass" to auth: {user:"user",pass:"pass"}

prompt user when stored couchdb password is incorrect

Removing config.json or changing the admin password on couchdb causes hoodie to no longer start due to invalid credentials. We should prompt the user to enter the new admin password and store that in config.json if successful.

Stop couchdb shutdown code from swallowing exit codes

Currently calls process.exit(0) every time

ctrl-c doesn’t stop couchdb

we do have code to intercept SIGINT, but it doesn’t seem to work.

Always serve index.html

Since hoodie is a front-end-only framework the best choice to build a big app using it, is using client-side-routing. The only problem is that the server doesn't always respond with the index.html.

If I request /foo/bar I want index.html served which then can handle the routing.

But if you request /js/hoodie.js you'll get hoodie.js, of course.

A good approach to implement this would either enabled it by default or using a config. For config we could go with the config-option built inside npm.

Built-in Update function

As far as I understand it, to update the backend of a hoodie app all we need is (or will be) npm update.

I wonder if it would be possible to run that command from within the server? I'd love to have a button in pocket saying "update hoodie" and everything gets updated and then the server restarted to simplify the update procedure of own hoodie installations

requesting a non-existing asset does not return 404

Let's say I've a hoodie app running at http://test.dev.
When I try to request an asset that does not exist, it end's up in a non-ending request:
http://test.dev/nothing-here.js

Prints 'undefined' to terminal when prompting for admin password

Interestingly, I've tracked this down to a call to the NPM config command which causes the subsequent read from stdin to output 'undefined' to the terminal. This should be fixed once #53 is done, but recording the issue here separately in case I'm mistaken about it's cause.

hoodie backend in proper

hoodie.js went pretty far in the past year, the frontend API is solid and the core features work. Still a bit buggy, but good enough to work with it, and we are building the first production apps on it already.

A problem we encountered is scaling. We were pretty pragmatic about about how to build the backend, to get it real fast™, which was great. But now's the time to make things more solid.

Be free on how you want to design the hoodie backend architecture. We have some suggestions, but we are sure so do you. You can build on what's already there or start from scratch. Just discuss your thoughts with @janl before building.

How it's working today (roughly)

Today's backend has a core module (hoodie-app) that starts each hoodie worker listed in the app's package.json

Each module has its own logic to listen to changes in databases. For example, if a hoodie app uses the users, shares and email module, all of them make separate continuous pull requests to every user database. That means we end up with hundreds of open connections pretty quickly, no matter whether the respective user is active or not.

How it could work tomorrow

Jan has put in the GET /_db_changes API in the CouchDB 1.4 trunk, we can use it for hoodie. Instead of keeping tons of requests open, Jan suggested a queue:

Hoodie core listens to /_db_changes. Each time there is a change for a database, we trigger events the modules can subscribe to (add|update|remove & add|update|remove:dbname). Then it loads the and triggers a docchanges & docchanges:dbname event with the response from GET /_changes?since= attached. It can also keep the connection open, for example for active users, until the queue is full. After that, each new request kills the one that was inactive for the longest time

A module like email would then listen to the docchanges events of the core module and and check if there are new email objects to be sent.

The heart of hoodie

At the core of hoodie, there are two things.

1. each user has its own database only she/he has access to
2. everything is based on synchronization for both, data and background task.

All local changes are pushed to the users CouchDB using the POST /_bulk_docs API (with new_edits=false flag) and pulled using long polling GET /_changes requests.

When he creates a task object using hoodie.store.add("task", {title: "remember the milk"}), the object being pushed to the couch will look like this (simplified):

{
  "_id": "task/rndid123",
  "_rev": "1-rndhash456",
  "type": "task",
  "title": "remember the milk"
}

When he sends an email using hoodie.email.send({to: "[email protected]", subject: "Hey Joe!", body: "woot woot!"}) it will internally also just create an object, looking like this (simplified):

{
  "_id": "$email/rndid123",
  "_rev": "1-rndhash456",
  "type": "$email",
  "to": "[email protected]",
  "subject": "Hey Joe!",
  "body": "woot woot!"
}

Both are pushed with POST /_bulk_docs and remote updates are recieved using GET /_changes. The frontend API just hides that away to let the user do things like

hoodie.email.send({…})
.then( onEmailSent, onEmailError)

So no matter how the CouchDB/Node.js backend will look like, the main requirement from the fronted is to

1. create a user database when signing up a new user
2. handle background jobs based on special objects synced to the user's database

ECONNREFUSED after I close my laptop

Running locally my hoodie app is amazeballs!
And then I close my laptop.
And then later I open it up, and I get
stuff that looks like this in my console: http://cl.ly/image/2f0P2b0U1R3K

The response from the call to _api/ is:

An error has occurred: {"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}

do not require www folder

hoodie start crashes, if www/index.html does not exist.

Would be nice if we'd check if it exists and only open browser in case it does.
In my particular use case, I only want the hoodie api, not the static server.

[static-server] add support for history.pushState()

All the cool kids do history.pushState() in their fancy HTML5 apps. Would be nice if we could support that out of the by a catch-all url rewrite.

The simplest way to do that would be: if there is no static asset at current path, return index.html. A smarter rewrite rule would be: every path that does not have a (known) extension, returns the index.html