krakenjs / kappa

A hierarchical npm-registry proxy

kappa

Based on [npm-delegate] (https://npmjs.org/package/npm-delegate) by Jason Denizac [email protected], this module is a hapi plugin used to proxy npm to support private npm repos without replicating the entire public registry.

NOTE: The config.json described below is a Hapi Composer manifest that describes kappa as a plugin. See ./example/global/config.json or ./example/local/config.json for more information.

Quickstart

TL;DCO (too long; didn't check out)

Using docker-compose, just run docker-compose up in a directory containing our docker-compose.yml file. Soon after you'll have a working kappa instance set up to use a local couch instance (falling back to the public registry) on 0.0.0.0:8000.

Global Installation

$ npm install -g kappa
$ kappa -c config.json

Local Installation

If you choose to install locally, running kappa is as easy as

$ npm init
$ npm install --save kappa

# add start script to package.json:
#    "scripts": {
#        "start": "kappa -c config.json",
#    }

$ npm start

You can then put those artifacts (config.json and package.json) under source control for simple deployments later:

$ git clone [email protected]:me/myregistry.git
$ cd myregistry
$ npm install
$ npm start

Options

kappa plugin currently supports the following parameters

• vhost - the virtual host associated with the kappa server, e.g. 'npm.mydomain.com'
• paths (optional) - any ordered array of npm repositories to use, e.g. Defaults to ['http://localhost:5984/registry/_design/app/_rewrite/', 'https://registry.npmjs.org/']
• rewriteTarballs (optional) - When true rewrites the tarball URL in packages to download each resource via kappa. When false, tarball URLs are left untouched, allowing the client to download package tarballs directly from the registry that fulfilled the package request. Defaults to true.

For read operations (GET, HEAD, etc) the proxy will first attempt to fetch the module from the first registry. If the requested module is not found it continues to the next registry, and so on.

For write operations the proxy will only attempt to write to the FIRST registry. All auth occurs with the first registry as well.

FAQs

I'm seeing npm install fail after a couple of minutes with: npm ERR! shasum check failed for ...

This is likely a result of the download exceeding node's default socket idle timeout. The best way to address this is by disabling the socket timeout for your server in the manifest.

{
    "host": "localhost",
    "port": 8000,
    "options": {
        "timeout": {
            "socket": false
        }
    }
}

kappa's Issues

npm ERR! shasum check failed

Using the latest kappa, installed on an EC2 server running under Docker (burkostya/npm-registry). We're consistently getting the "npm ERR! shasum check failed" error on random packages proxied from npmjs.org. I did an npm cache clean on clients, set the timeout option on the server (and restarted), and we still get the error on virtually all npm installs. Any hints?

Add request id to proxy logs

This data set should include request id

Enable backpressure detection

Hapi provides some facilities to deal with load, so we should be taking advantage of them.

Note: this is really most likely going to be done via server config json since kappa is just a plugin, but at the very least it should be added to the example config.

Duplicate _rewrite in tarball path breaks install

I can run npm info and npm publish using a config where the first path is "http://localhost:5984/registry/_design/app/_rewrite/", but when I try to npm install against the kappa registry, it fails with 404 Not Found on the tarball. The tarball path in the info output has the kappa hostname rewritten (from util.rewrite), but the /registry/_design/app/_rewrite/ path is still there. That then gets duplicated when the path to the first server is built. The couchdb log for the local npmjs looks like
GET /registry/_design/app/_rewrite/registry/_design/app/_rewrite/<package-name>/... .tgz 404

I think the util.rewriter function needs to strip that portion of the path. Any reason why it shouldn't?

exports.rewriter = function rewriter(protocol, hostname, port) {
    return function rewrite(path) {
        if (typeof path === 'string') {
            path = url.parse(path);
            path.protocol = protocol;
            path.host = path.hostname = hostname;
            path.port = port;
            path.path = path.path.replace( /\/registry\/_design\/app\/_rewrite/, '' );
            path.pathname = path.path;
            path = path.format();
        }
        return path;
    };
};

Log IP addresses

The IPs aren't appearing in the logs. Let's add them so we have better insight into usage.

Will kappa still be maintained after private modules are released for NPM?

I saw NPM is releasing private-modules so people can maintain their own private npm registries. I was wondering if this would affect kappa at all? Will it still be maintained? Or perhaps they are not related at all and address entirely different use-cases haha.

Please label this as question.

Better handling of registry paths

Currently, paths require trailing slashes and without them the app fails in non-obvious ways. Update logic to deal with missing trailing slashes.

Unexpected token when running kappa through docker-compose

Steps to reproduce

• Clone the kappa repository
• Run docker-compose up (in my case the IP of VMware is 192.168.9.130)
• Create a demo directory, e.g. ~/foo
• Switch to that directory, i.e. cd ~/foo
• Try to install express using kappa: npm install express --registry http://192.168.9.130:8000

Expected result

• kappa does not find express in the local registry
• kappa fetches express from the official npm registry

Actual result

• kappa does not find express in the local registry
• kappa throws a few exceptions (the details of the exceptions are shown below)

Details of the exceptions

[…]
registry_1 | [error] [<0.105.0>] OS Process Error <0.2197.0> :: {not_found,
registry_1 |                                                     <<"document not found">>}
registry_1 | [info] [<0.105.0>] 172.18.0.3 - - GET /registry/_design/app/_show/package/express?pkg=express 404
kappa_1    | 160525/073826.927, internalError, message: Unexpected token  stack: SyntaxError: Unexpected token 
kappa_1    |     at Object.parse (native)
kappa_1    |     at Object.internals.tryParseBuffer (/usr/lib/node_modules/kappa/node_modules/wreck/lib/index.js:467:25)
kappa_1    |     at finish (/usr/lib/node_modules/kappa/node_modules/wreck/lib/index.js:308:28)
kappa_1    |     at wrapped (/usr/lib/node_modules/kappa/node_modules/hoek/lib/index.js:866:20)
kappa_1    |     at onReaderFinish (/usr/lib/node_modules/kappa/node_modules/wreck/lib/index.js:346:16)
kappa_1    |     at g (events.js:260:16)
kappa_1    |     at emitNone (events.js:72:20)
kappa_1    |     at emit (events.js:166:7)
kappa_1    |     at finishMaybe (_stream_writable.js:468:14)
kappa_1    |     at endWritable (_stream_writable.js:478:3)
kappa_1    | 160525/073826.927, request, http://0.0.0.0:8000: get /express {} 500 (1174ms) 
kappa_1    | 160525/073837.594, ops, memory: 34Mb, uptime (seconds): 1906.072, load: 0.0029296875,0.0146484375,0.04541015625
registry_1 | [error] [<0.2187.0>] OS Process Error <0.2197.0> :: {not_found,
registry_1 |                                                      <<"document not found">>}
registry_1 | [info] [<0.2187.0>] 172.18.0.3 - - GET /registry/_design/app/_show/package/express?pkg=express 404
kappa_1    | 160525/073838.115, internalError, message: Unexpected token  stack: SyntaxError: Unexpected token 
kappa_1    |     at Object.parse (native)
kappa_1    |     at Object.internals.tryParseBuffer (/usr/lib/node_modules/kappa/node_modules/wreck/lib/index.js:467:25)
kappa_1    |     at finish (/usr/lib/node_modules/kappa/node_modules/wreck/lib/index.js:308:28)
kappa_1    |     at wrapped (/usr/lib/node_modules/kappa/node_modules/hoek/lib/index.js:866:20)
kappa_1    |     at onReaderFinish (/usr/lib/node_modules/kappa/node_modules/wreck/lib/index.js:346:16)
kappa_1    |     at g (events.js:260:16)
kappa_1    |     at emitNone (events.js:72:20)
kappa_1    |     at emit (events.js:166:7)
kappa_1    |     at finishMaybe (_stream_writable.js:468:14)
kappa_1    |     at endWritable (_stream_writable.js:478:3)
kappa_1    | 160525/073838.115, request, http://0.0.0.0:8000: get /express {} 500 (766ms) 
kappa_1    | 160525/073852.599, ops, memory: 35Mb, uptime (seconds): 1921.078, load: 0.0029296875,0.0146484375,0.04541015625
kappa_1    | 160525/073907.605, ops, memory: 35Mb, uptime (seconds): 1936.083, load: 0.0029296875,0.0146484375,0.04541015625
kappa_1    | 160525/073922.607, ops, memory: 35Mb, uptime (seconds): 1951.084, load: 0.0029296875,0.0146484375,0.04541015625
kappa_1    | 160525/073937.615, ops, memory: 34Mb, uptime (seconds): 1966.093, load: 0.0029296875,0.0146484375,0.04541015625
registry_1 | [error] [<0.106.0>] OS Process Error <0.2197.0> :: {not_found,
registry_1 |                                                     <<"document not found">>}
registry_1 | [info] [<0.106.0>] 172.18.0.3 - - GET /registry/_design/app/_show/package/express?pkg=express 404
kappa_1    | 160525/073938.888, internalError, message: Unexpected token  stack: SyntaxError: Unexpected token 
kappa_1    |     at Object.parse (native)
kappa_1    |     at Object.internals.tryParseBuffer (/usr/lib/node_modules/kappa/node_modules/wreck/lib/index.js:467:25)
kappa_1    |     at finish (/usr/lib/node_modules/kappa/node_modules/wreck/lib/index.js:308:28)
kappa_1    |     at wrapped (/usr/lib/node_modules/kappa/node_modules/hoek/lib/index.js:866:20)
kappa_1    |     at onReaderFinish (/usr/lib/node_modules/kappa/node_modules/wreck/lib/index.js:346:16)
kappa_1    |     at g (events.js:260:16)
kappa_1    |     at emitNone (events.js:72:20)
kappa_1    |     at emit (events.js:166:7)
kappa_1    |     at finishMaybe (_stream_writable.js:468:14)
kappa_1    |     at endWritable (_stream_writable.js:478:3)
kappa_1    | 160525/073938.888, request, http://0.0.0.0:8000: get /express {} 500 (848ms) 
[…]

npm add owner

I cannot get npm add owner to work with kappa. Anybody any idea ? Is this supposed to work ? Can I somehow do this manually with clever couchdb calls ?

C:\Development\iris-forms>npm owner add matthiasg "iris-forms"                  
npm ERR! owner mutate Error getting user data for matthiasg                     
npm ERR! Error: unauthorized Authentication required.: -/user/org.couchdb.user:m
atthiasg                                                                        
npm ERR!     at RegClient.<anonymous> (C:\Program Files\nodejs\node_modules\npm\
node_modules\npm-registry-client\lib\request.js:300:14)                         
npm ERR!     at C:\Program Files\nodejs\node_modules\npm\node_modules\npm-regist
ry-client\lib\request.js:243:7                                                  
npm ERR!     at Gunzip.onEnd (zlib.js:166:5)                                    
npm ERR!     at Gunzip.emit (events.js:117:20)                                  
npm ERR!     at _stream_readable.js:943:16                                      
npm ERR!     at process._tickCallback (node.js:419:13)                          
npm ERR! If you need help, you may report this *entire* log,                    
npm ERR! including the npm and node versions, at:                               
npm ERR!     <http://github.com/npm/npm/issues>                                 

npm ERR! System Windows_NT 6.2.9200                                             
npm ERR! command "C:\\Program Files\\nodejs\\\\node.exe" "C:\\Program Files\\nod
ejs\\node_modules\\npm\\bin\\npm-cli.js" "owner" "add" "matthiasg" "iris-forms" 
npm ERR! cwd C:\Development\iris-forms                                          
npm ERR! node -v v0.10.31                                                       
npm ERR! npm -v 1.4.23                                                          
npm ERR!                                                                        
npm ERR! Additional logging details can be found in:                            
npm ERR!     C:\Development\iris-forms\npm-debug.log                            
npm ERR! not ok code 0                                                          

I am executing this from the original publishers account and the publish call right before this worked fine. So the npm whoami user has owner access, but i cannot add user matthiasg even though that user exists in the private registry against which we are publishing.

Can't npm install?

kappa.conf.json

{
        "servers": [
            {
                "host": "localhost",
                "port": 8080
            }
        ],
        "plugins": {
            "kappa": {
                "vhost": "localhost",
                "paths": [
                    "http://alex:foo@localhost:5984",
                    "https://registry.npmjs.org/"
                ]
            }
        }
    }

package.json

{
  "name": "private-test",
  "version": "0.5.6",
  "description": "",
  "dependencies": {},
  "devDependencies": {
    "gulp-symlink": "*",
    "coffee-script": "*",
    "coffee-errors": "*",
    "sinon-chai": "*",
    "chai": "*",
    "sinon": "*",
    "mocha": "*"
  }
}

cmd

$ npm --registry=http://localhost:8080 install

output

npm WARN package.json [email protected] No description
npm WARN package.json [email protected] No README data
npm http GET http://localhost:8080/sinon-chai
npm http GET http://localhost:8080/coffee-script
npm http GET http://localhost:8080/coffee-errors
npm http GET http://localhost:8080/mocha
npm http GET http://localhost:8080/gulp-symlink
npm http GET http://localhost:8080/sinon
npm http GET http://localhost:8080/chai
npm http 304 http://localhost:8080/sinon-chai
npm http 304 http://localhost:8080/coffee-script
npm http GET http://localhost/sinon-chai/-/sinon-chai-2.5.0.tgz
npm ERR! fetch failed http://localhost/sinon-chai/-/sinon-chai-2.5.0.tgz
npm http 304 http://localhost:8080/mocha
npm http GET http://localhost/mocha/-/mocha-1.17.1.tgz
npm ERR! fetch failed http://localhost/mocha/-/mocha-1.17.1.tgz
npm http 304 http://localhost:8080/coffee-errors
...

wondering if this is NPM issue or kappa, it appears the port is being lost?

Switch metrics over to `good`

Just started looking into this, but kappa should switch stats/metrics to use spumko/good plugin goodness.

Since they're both just plugins this may only be a sample config change, but investigate whether kappa can detect the existence of good and log custom metrics data.

opt out of tarball rewrites

It would be nice to be able to disable tarball rewrites.

Trouble with default local registry in README

I followed the README for "paths" option in config.json
"paths": [ "http://localhost:5984/registry/_design/ghost/_rewrite/", ...]
and couldn't read from local. I also tried the following which matches my npm config
"paths": [ "http://localhost:5984/registry/_design/app/_rewrite/", ...]
but that didn't work either. However, the following works beautifully.
"paths": [ "http://localhost:5984/registry/", ...]
Has something changed in the code to make this shortened version work and/or is there something else I've configured that dictates this form? I'm not using the registry vhost'ing that allows https://registry.npmjs.org to work. Either way, we should clarify the README.

Thanks! Love what you've done with kappa.

HTTPS Parse Error

I'm using node v0.10.28, and I get this error for any https connections.
I have tested my certificate and key with a simple node https server, and it works as expected.

Update kappa example to work with newest version of `good`

The current example reflects version ^3, which is no longer latest.

gzip not supported?

recently the chinese npm mirror https://registry.npm.taobao.org enabled gzip, then our kappa server got 500 when proxy to this site.

here's our kappa.json:

{
    "servers": [
        {
            "host": "127.0.0.1",
            "port": "7000"
        }
    ],
    "plugins": {
        "kappa": {
            "vhost": "npmcc.creditcloud.com",
            "rewriteTarballs": true,
            "paths": [
                "http://127.0.0.1:7001/",
                "https://registry.npm.taobao.org/",
                "https://registry.npmjs.org/"
            ]
        }
    }
}

do request with Accept-Encoding: gzip

node@NODE:~$ curl http://127.0.0.1:7000/watchify -H 'Host: npmcc.creditcloud.com' --compressed
{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred"}node@NODE:~$

Add field to json proxied, showing which its original registry was.

I'd love to make sure we can tell internal and external registry information apart.

public hostname issue

Hey hey, I'm trying to run kappa inside of docker (https://github.com/terinjokes/docker-npmjs), but the problem I'm having is that kappa fails to proxy if you use an external hostname, however when you provide an internal hostname (that a docker container gives you) it tries to send that back to npm so all tarball requests fail

npm adduser failing

Hi!
So, I was trying to npm adduser and I got some great errors.

Couch says:
[info] [<0.173.0>] 127.0.0.1 - - PUT /registry/_design/ghost/_rewrite/-/user/org.couchdb.user:parris 404

Kappa reports:

2014-01-09T08:20:16.493Z INFO,REQUEST 127.0.0.1 PUT /-/user/org.couchdb.user:parris 404

npm adduser:

npm adduser
Username: parris
Password: 
Email: [email protected]
npm http PUT http://localhost:1337/-/user/org.couchdb.user:parris
npm http 404 http://localhost:1337/-/user/org.couchdb.user:parris
npm ERR! Error: not_found missing: -/user/org.couchdb.user:parris
npm ERR!     at RegClient.<anonymous> (/Users/parris/.nvm/v0.10.23/lib/node_modules/npm/node_modules/npm-registry-client/lib/request.js:272:14)
npm ERR!     at Request.self.callback (/Users/parris/.nvm/v0.10.23/lib/node_modules/npm/node_modules/request/request.js:123:22)
npm ERR!     at Request.EventEmitter.emit (events.js:98:17)
npm ERR!     at Request.<anonymous> (/Users/parris/.nvm/v0.10.23/lib/node_modules/npm/node_modules/request/request.js:877:14)
npm ERR!     at Request.EventEmitter.emit (events.js:117:20)
npm ERR!     at IncomingMessage.<anonymous> (/Users/parris/.nvm/v0.10.23/lib/node_modules/npm/node_modules/request/request.js:828:12)
npm ERR!     at IncomingMessage.EventEmitter.emit (events.js:117:20)
npm ERR!     at _stream_readable.js:920:16
npm ERR!     at process._tickCallback (node.js:415:13)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR!     <http://github.com/isaacs/npm/issues>

npm ERR! System Darwin 12.5.0
npm ERR! command "/Users/parris/.nvm/v0.10.23/bin/node" "/Users/parris/.nvm/v0.10.23/bin/npm" "adduser"
npm ERR! cwd /Volumes/eb_home/work/eb-npm-registry
npm ERR! node -v v0.10.23
npm ERR! npm -v 1.3.17
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /Volumes/eb_home/work/eb-npm-registry/npm-debug.log
npm ERR! not ok code 0

Here is my setup: https://github.com/parris/eb-npm-registry/ and a readme about all the other stuff I did.

Only delete host for registries > 0

The first registry needs to keep the host header such that npm can create the correct tarball URL via // legacy kludge

Need doc on how to increase Hapi payload.maxBytes

If a tarball is greater than 1mb default payload maxBytes, publish will fail.
2014-03-05T23:56:58.212Z ERROR Payload content length greater than maximum allowed: 1048576

This config.json shows the server options to increase it.

{
    "servers": [ {
       "options": { "payload": { "maxBytes": 104857600 } },
       "port": 80
     } ],
    "plugins": { "kappa": { ...  }  }
}

Rewrite tarball URLs to always go through kappa/internal host

Currently, all tarball urls from public repo include the hostname registry.npmjs.org, thus once the initial json payload is downloaded via kappa, subsequent tarball requests are made of the public repo. This change will funnel all traffic through kappa, but more importantly through the private npm host/dns, such that caching can be added to the pipeline without messing with registry.npmjs.org via dns.

So instead of:

client --> kappa --> private couchdb -\
     \---------------------------------\---> public repo

you could support

client --> kappa --> private couchdb --> private cache --> public repo

Update to work with hapi@8

Hapi 8 is GA now so we should support it.

unref stats timer

Until we get around to revisiting the statistics stuff, add unref() to setTimeout opaque value such that it doesn't keep the process alive when everything else is closed.

npm search supposed to work?

I've installed https://github.com/npm/npm-registry-couchapp, and pointed kappa to the replicated skim registry.

Npm install works, but npm search:

iliakan> npm search kappa
npm WARN Building the local index for the first time, please be patient
npm http request GET http://registry.javascript.ru:5555/-/all
npm http 504 http://registry.javascript.ru:5555/-/all
npm http request GET http://registry.javascript.ru:5555/-/all
npm http 504 http://registry.javascript.ru:5555/-/all
npm http request GET http://registry.javascript.ru:5555/-/all
npm http 504 http://registry.javascript.ru:5555/-/all
npm ERR! Darwin 13.4.0
npm ERR! argv "node" "/usr/local/bin/npm" "search" "kappa"
npm ERR! node v0.11.14
npm ERR! npm  v2.0.0
npm ERR! code E504

npm ERR! Gateway Time-out : -/all

Is it normal?

Install kappa behind a corporate proxy

Is there any way to install kappa behind a corporate proxy ?

I sucessfully installed kappa with only one declared paths to my local couchdb instance (tested commands : npm {adduser, search, test, publish, install}), but npm install XXX fails (freeze) if I add http://registry.npmjs.org/ as a second paths to the kappa configuration file.

I also tried to run kappa with some options without more success :

export http_proxy="http://user:password@host:port"
export https_proxy=$http_proxy
# this works well
wget http://registry.npmjs.org/

# Setting the proxy configuration to npm
npm config set proxy $http_proxy
npm config set https-proxy $http_proxy

# npm client fails with this configuration
kappa -c config.json --proxy $http_proxy --https-proxy $https_proxy

Usable in read-only mode

Hi,

This is just a suggestion. I went over the docs, opened and closed issues and haven't seen this mentioned.
I'd love to be able to use it without requiring ever writing to disk. This would allow this to be run on something like Heroku, I think if that were possible, so many people would use it like that.

To achieve that, I guess you either rely on a different storage, or... publish package changes as commits on the hosting's version control and kappa pulls the packages from there.

It sounds a little clunky, but I wouldn't mind it personally, most private repos aren't hosting tenths of packages.

Thanks

Where to find documentation?

I am currently trying to get kappa up and running, and I wonder where the documentation for this module is.

Of course we have the README.md, but e.g., it does not describe how to configure kappa.

Is there some more documentation available? If so, where?

patch .tarball on response

where would I add that? I can't waste time on setting this up anymore haha, just going to rewrite .tarball on the way out so it matches the subdomain, it keeps barfing on different stuff, it would be nice to just add http://localhost:5984/registry/_design/app/_rewrite/ to kappa's config

oh wut, hapi has a request library built in? that's odd, digging into that to see how to work with this

Futon access

Might be a good idea to prevent Futon access through the proxy, or allow the option to do it somehow. If someone binds their couchdb to only loopback, kappa would expose the next proxy's Futon in the list, including the public registry's.

Here is what I did in a wrapper hapi plugin:

// Prevent access to CouchDB Futon through proxy
function forbidden(req, reply) {
    reply('{"error":"forbidden","reason":"public access is not allowed"}')
        .code(403)
        .type('application/json')
        .header('Cache-Control', 'no-cache');
}

['GET', 'POST', 'PUT', 'DELETE', '*'].forEach(function (method) {
    plugin.route({
    method: method,
    path: '/_utils/{p*}',
        config: {
            handler: forbidden
        }
    });
});

HEAD Requests

Hey @jeffharrell, Scott from Dow Jones

Does kappa support HEAD requests?

I see it in the docs but not the plugin code. I have a local instance that throws 500s on HEAD and our company instance just hangs. I ask because we are migrating to artifactory and they check all the packages with HEAD before proceeding (and of course NPM supports HEAD).

Undefined is not a function in Good Module

On node: v0.12.2

Commands I run:
npm init {follow instructions}
npm install kappa --save
npm start

$ npm start
> [email protected] start /home/kridgway/kappa
> kappa -c config.json

/home/kridgway/kappa/node_modules/good/lib/index.js:12
    server.on('stop', function () {
           ^
TypeError: undefined is not a function
    at exports.register (/home/kridgway/kappa/node_modules/good/lib/index.js:12:12)
    at /home/kridgway/kappa/node_modules/kappa/node_modules/hapi/lib/plugin.js:187:23
    at iterate (/home/kridgway/kappa/node_modules/kappa/node_modules/hapi/node_modules/items/lib/index.js:35:13)
    at Object.exports.serial (/home/kridgway/kappa/node_modules/kappa/node_modules/hapi/node_modules/items/lib/index.js:38:9)
    at internals.Plugin.register (/home/kridgway/kappa/node_modules/kappa/node_modules/hapi/lib/plugin.js:171:11)
    at /home/kridgway/kappa/node_modules/kappa/node_modules/hapi/node_modules/glue/lib/index.js:188:22
    at iterate (/home/kridgway/kappa/node_modules/kappa/node_modules/hapi/node_modules/items/lib/index.js:35:13)
    at Object.exports.serial (/home/kridgway/kappa/node_modules/kappa/node_modules/hapi/node_modules/items/lib/index.js:38:9)
    at /home/kridgway/kappa/node_modules/kappa/node_modules/hapi/node_modules/glue/lib/index.js:185:19
    at iterate (/home/kridgway/kappa/node_modules/kappa/node_modules/hapi/node_modules/items/lib/index.js:35:13)

My package.json:

{
  "name": "my-kappa",
  "version": "0.0.1",
  "description": "My Installation of Kappa, an NPM Proxy.\"",
  "main": "index.js",
  "dependencies": {
    "good": "^6.1.2",
    "good-console": "^5.0.0",
    "hapi": "^8.4.0",
    "kappa": "^1.0.0-rc.11"
  },
  "devDependencies": {},
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "kappa -c config.json"
  },
  "author": "",
  "license": "ISC"
}    

Does Kappa work on the latest Hapi/Good versions? In the interim, does anybody know what exact versions of the modules work to run Kappa?

Update hapi to 6.6.0

Kappa is now 4 major versions behind hapi. Are there any plans on upgrading, if so, what are the tasks that need to be done?

npm search should search in all registries

When I run npm search it only returns results for the first registry. I think it should behave in a way consistent with the behaviour of npm install: get all registries' /-/all/ and merge the output lists giving priority to the first registries.

HAPI unspecified error

when install some package, i always see HAPI,REQUEST,ERROR,ABORTED unspecified in logs:

014-09-01T09:43:49.424Z INFO,REQUEST 172.16.2.2 GET /mongodb/1.4.7 200
2014-09-01T09:43:50.504Z INFO,REQUEST 172.16.2.2 GET /xml2js/0.2.6 200
2014-09-01T09:43:51.816Z INFO,REQUEST 172.16.2.2 GET /ejs 200
2014-09-01T09:43:54.023Z HAPI,REQUEST,ERROR,ABORTED unspecified
2014-09-01T09:43:54.023Z HAPI,REQUEST,ERROR,CLOSED unspecified

so what's the error meaning.

Scope passthrough support

It looks like some modules, specifically related to angular 2, are using scopes.

https://angular.io/docs/ts/latest/quickstart.html

Feature request: To allow module downloads with scopes to pass through and immediately be re-directed to public npm.

Will see if I can send a PR soon

Migrate to Hapi 2.x

Current version of Good (#17) requires Hapi 2, and previous versions are not compatible with the Hapi version we're at.

paths vs servers

The README and defaults.json use paths, while the source uses servers

Document increasing (or disabling) socket timeout.

User on a slow network was getting a truncated tgz for a particularly large module. Need to document how to increase or disable idle socket timeouts.

Unable to get "good" working with kappa

Following the example config, no matter what version of good I try to install I get:

Error: Cannot find module 'good/lib/reporter'

Which makes sense since no version has a file called "reporter.js" available at that path.

However, after trying several configurations I am unable to get good working with Kappa.

Pull in versions of packages not in the local repo from the public repo

Suppose I have bower 1.0.0 in my local repo and I try to install bower version 2.0.0, which is present in the public repo, I get an error message saying only version 1.0.0 was found.

I should be able to get a newer version of a package from the public repo even when an older version of the same package is present in my local repo.

Seems to respond slower and slower over time.

When first launched, kappa responses pretty quickly. However as time goes on it gets pretty sluggish, causing installs that took seconds to turn into minutes.

I'm not used to debugging hapi. What should I do to look into this?

unable to publish

Sorry for the issue, but I'm pretty stuck. Any help would be fantastic.

My setup:

couch: http://admin:[email protected]:5984/registry/
kappa: http://npm.miketown3.com

I can hit something like npm.miketown3.com/mocha and I get the details from the main npm. Additionally, npm --registry "http://npm.miketown3.com" install works if I only depend on public modules from the main repo. However, npm --registry "http://npm.miketown3.com" adduser results in the below

npm http PUT http://npm.miketown3.com/-/user/org.couchdb.user:chevett
npm http 409 http://npm.miketown3.com/-/user/org.couchdb.user:chevett
npm http GET http://npm.miketown3.com/-/user/org.couchdb.user:chevett

The last GET just hangs and never completes.

So, I tried hitting couch directly:

npm --registry "http://admin:[email protected]:5984/registry/_design/app/_rewrite" adduser

And that works just fine... I can publish and install using that couch url, but of course I don't get anything from the main npm...

My kappa config is like so:

{
    "servers": [
        {
            "port": 80
        }
    ],
    "plugins": {
        "kappa": {
            "vhost": "npm.miketown3.com",
            "paths": [
                "http://admin:password@localhost:5984/registry/_design/app/_rewrite",
                "https://registry.npmjs.org/"
            ]
        }
    }
}

My whole kappa repo is here: https://github.com/chevett/mpm

Any ideas? Thanks!

Question: NPM CLI setup for Kappa

I'm probably a complete noob, but, what am I missing here? Am I missing that I need to setup a registry backing Kappa? https://github.com/rvagg/npmjs.org/blob/6793c71165934423f1989f54a754a6691a94f3b0/README.md

On the Kappa instance itself, I can

npm --registry=http://localhost:8000 install lodash

And I at least see in the running Kappa instance, it 404 on this request.

150506/195745.123, request, http://localhost:8000: get /lodash {} 404 (9ms)

If I try on my local machine...

npm set registry http://my-kappa-box.mynetwork.com:8000 install request
npm install

Error log

$ npm install
npm WARN package.json [email protected] No description
npm WARN package.json [email protected] No repository field.
npm WARN package.json [email protected] No README data
npm ERR! Error: connect ECONNREFUSED
npm ERR!     at errnoException (net.js:904:11)
npm ERR!     at Object.afterConnect [as oncomplete] (net.js:895:19)
npm ERR!  { [Error: connect ECONNREFUSED]
npm ERR!   code: 'ECONNREFUSED',
npm ERR!   errno: 'ECONNREFUSED',
npm ERR!   syscall: 'connect',
npm ERR!   parent: 'synacor-application-fe-primetime-assets' }
npm ERR!
npm ERR! If you are behind a proxy, please make sure that the
npm ERR! 'proxy' config is set properly.  See: 'npm help config'

npm ERR! System Darwin 14.0.0
npm ERR! command "node" "/usr/local/bin/npm" "install"
npm ERR! cwd /Users/kridgway/Code/myproject
npm ERR! node -v v0.10.29
npm ERR! npm -v 1.4.21
npm ERR! syscall connect
npm ERR! code ECONNREFUSED
npm ERR! errno ECONNREFUSED
npm ERR! stack Error: connect ECONNREFUSED
npm ERR! stack     at errnoException (net.js:904:11)
npm ERR! stack     at Object.afterConnect [as oncomplete] (net.js:895:19)
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /Users/kridgway/Code/myproject/npm-debug.log
npm ERR! not ok code 0

Further, once I figure out my issue, is there a way to set in our package.json for the myproject we're working on, so that whenever you 'npm install', within the project that has package.json, it will always try our local kappa instance? Or does a person always have to 'npm set registry' first?

Any help is appreciated for helping me understand the setup end-to-end.

Fallback is not happening in Kappa,

Fallback is not happening in Kappa, I am using the following config file and I am able to install the npm packages on the first repository(http://privaterepo/lgc) but i am not able to install any packages from the second repository. I was trying install bytes npm package which is not available in the first repository and I am expecting it to be installed from second repository (means it should fallback to second repository if it is not available in first repository) . I would your appreciate your help on this

{
"servers": [
{
"host": "localhost",
"port": 9500,
"options": {
"timeout": {
"socket": false
} ,
"payload": {
"maxBytes": 10000000
}
}
}
],
"plugins": {
"kappa": {
"paths": [
"http://privaterepo/lgc",
"https://registry.npmjs.org/"

        ]
    }
}

}

Only responds to requests from localhost

I am using kappa inside of a vagrant vm, and it will respond correctly to requests from within the vm but closes the connection without a response when requests come from outside the vm. I have confirmed that port forwarding is working correctly in vagrant using netcat.

My config.json file is:

{
    "servers": [
        {
            "host": "localhost",
            "port": "8080"
        }
    ],
    "plugins": {
        "kappa": {
            "paths": [
                "http://localhost:5984/registry/_design/app/_rewrite/",
                "https://registry.npmjs.org/"
            ]
        }
    }
}

and this is the result of issuing a GET request from the host machine:

$ curl --verbose http://localhost:8080
* Adding handle: conn: 0x7ff67b804000
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7ff67b804000) send_pipe: 1, recv_pipe: 0
* About to connect() to localhost port 8080 (#0)
*   Trying ::1...
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.30.0
> Host: localhost:8080
> Accept: */*
> 
* Empty reply from server
* Connection #0 to host localhost left intact
curl: (52) Empty reply from server

I imagine that I'm configuring something incorrectly, but I'd really appreciate any pointers on this. Thanks in advance!

Make registry timeout configurable

Also, ensure timeout doesn't fail, but allows for fallback.

Explicitly log errors from proxied requests.

The most appropriate place may be in onError

vhost issue + NPM install shasum check failed

After updating (maybe from RC 8), vhost appears to be broken. I get the checksum error because kappa is not writing the vhost, and npm is attempting to download http://localhost:8200/something/-/something-2.0.4.tgz, while the server I am installing on is serving a page for port 8200.

Add responding registry to log

When a package GET is run, I'd like to get an indication whether the package was found in the first registry. I envision a more general logging of which registry (by url/name or index number) served the package, but just an indication of any iteration would work.