Can't install geoip-lite about node-geoip

[Tarang]

I also get something similar on Ubuntu.

It works like this way:

git clone https://github.com/bluesmoon/node-geoip
cd node-geo
npm install
node scripts/updatedb.js

It doesn't work like this

npm install geoip-lite

Retrieving GeoIPCountryCSV.zip ... DONE
Extracting GeoIPCountryCSV.zip ... DONE
Processing Data (may take a moment) ... DONE

Retrieving GeoIPv6.csv ... DONE
Processing Data (may take a moment) ... DONE

Retrieving GeoLiteCity-latest.zip ... DONE
Extracting GeoLiteCity-latest.zip ...Killed

npm ERR! [email protected] updatedb: `node scripts/updatedb.js`
npm ERR! Exit status 137
npm ERR! 
npm ERR! Failed at the [email protected] updatedb script.
npm ERR! This is most likely a problem with the geoip-lite package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node scripts/updatedb.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls geoip-lite
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 3.11.0-12-generic
npm ERR! command "/usr/bin/node" "/usr/bin/npm" "run-script" "updatedb"
npm ERR! cwd /root/test/node_modules/geoip-lite
npm ERR! node -v v0.10.24
npm ERR! npm -v 1.3.21
npm ERR! code ELIFECYCLE
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /root/test/node_modules/geoip-lite/npm-debug.log
npm ERR! not ok code 0
npm ERR! [email protected] postinstall: `npm run-script updatedb`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] postinstall script.
npm ERR! This is most likely a problem with the geoip-lite package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     npm run-script updatedb
npm ERR! You can get their info via:
npm ERR!     npm owner ls geoip-lite
npm ERR! There is likely additional logging output above.

npm ERR! System Linux 3.11.0-12-generic
npm ERR! command "/usr/bin/node" "/usr/bin/npm" "install" "https://github.com/bluesmoon/node-geoip/archive/master.tar.gz"
npm ERR! cwd /root/test
npm ERR! node -v v0.10.24
npm ERR! npm -v 1.3.21
npm ERR! code ELIFECYCLE
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /root/test/npm-debug.log
npm ERR! not ok code 0

In the npm debug log

1686 verbose rebuildBundles   'lazy',
1686 verbose rebuildBundles   'rimraf',
1686 verbose rebuildBundles   'unzip' ]
1687 info install [email protected]
1688 info postinstall [email protected]
1689 verbose unsafe-perm in lifecycle false
1690 info [email protected] Failed to exec postinstall script
1691 info /root/test/node_modules/geoip-lite unbuild
1692 info preuninstall [email protected]
1693 info uninstall [email protected]
1694 verbose true,/root/test/node_modules,/root/test/node_modules unbuild [email protected]
1695 info postuninstall [email protected]
1696 error [email protected] postinstall: `npm run-script updatedb`
1696 error Exit status 1
1697 error Failed at the [email protected] postinstall script.
1697 error This is most likely a problem with the geoip-lite package,
1697 error not with npm itself.
1697 error Tell the author that this fails on your system:
1697 error     npm run-script updatedb
1697 error You can get their info via:
1697 error     npm owner ls geoip-lite
1697 error There is likely additional logging output above.
1698 error System Linux 3.11.0-12-generic
1699 error command "/usr/bin/node" "/usr/bin/npm" "install" "https://github.com/bluesmoon/node-geoip/archive/master.tar.gz"
1700 error cwd /root/test
1701 error node -v v0.10.24
1702 error npm -v 1.3.21
1703 error code ELIFECYCLE
1704 verbose exit [ 1, true ]

[bluesmoon]

These are two different issues.

@soichih your issue, the ENOSPC error, means that your filesystem is either full (no space left), or if you have a unix like system, then it has run out of inodes. See here for more info: http://www.noah.org/wiki/ENOSPC_error,_yet_'df'_shows_available_space

[bluesmoon]

@Tarangp I cannot reproduce your issue. This is what I get:

> [email protected] postinstall /Users/philip/Projects/node_modules/geoip-lite
> npm run-script updatedb


> [email protected] updatedb /opt/local/src/node_modules/geoip-lite
> node scripts/updatedb.js

Retrieving GeoIPCountryCSV.zip ... DONE
Extracting GeoIPCountryCSV.zip ... DONE
Processing Data (may take a moment) ... DONE

Retrieving GeoIPv6.csv ... DONE
Processing Data (may take a moment) ... DONE

Retrieving GeoLiteCity-latest.zip ... DONE
Extracting GeoLiteCity-latest.zip ... DONE
Processing Data (may take a moment) ...
Still working (69191) ...
Still working (179076) ...
Still working (269654) ...
Still working (330818) ...
Still working (448607) ...
Still working (563976) ...
Still working (674358) ...
Still working (792452) ...
Still working (900232) ...
Still working (1002795) ...
Still working (1109617) ...
Still working (1224685) ...
Still working (1340982) ...
Still working (1460285) ...
Still working (1567188) ...
Still working (1672106) ...
Still working (1794392) ... DONE

Retrieving GeoLiteCityv6.csv ... DONE
Processing Data (may take a moment) ... DONE


Successfully Updated Databases from MaxMind.
[email protected] ../node_modules/geoip-lite
├── [email protected] ([email protected])
└── [email protected] ([email protected], [email protected], [email protected])

There doesn't seem to be a problem with the postinstall script. Could it be that the script only got a partial download of the csv?

[soichih]

I am pretty positive that my disk space / inodes were ok.. But anyway, I just tried "npm install geoip-lite" and it worked fine this time. I am not sure what changed, but it was some transient issue.

[Tarang]

@bluesmoon try on a fresh Ubuntu 13.10 x64 DigitalOcean instance

sudo apt-get install python-software-properties python g++ make
sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs

npm install geoip-lite

Retrieving GeoIPCountryCSV.zip ... DONE
Extracting GeoIPCountryCSV.zip ... DONE
Processing Data (may take a moment) ... DONE

Retrieving GeoIPv6.csv ... DONE
Processing Data (may take a moment) ... DONE

Retrieving GeoLiteCity-latest.zip ... DONE
Extracting GeoLiteCity-latest.zip ...Killed
...
(Same error details in previous post)

The error comes from extraction since the "DONE" isn't written

[patelatharva]

I am facing the same issue as @Tarangp has mentioned while running on Mac OS X. Can using a newer version of 'unzip' package or using some other library like 'zlib' be helpful to give a try?!

[bluesmoon]

It would be helpful if either of you could trace through the code and find out what's going on. I cannot reproduce this. My best guess is that the Retrieving step only got a partial GeoLiteCity-latest.zip file, and that's causing the unzip to fail.

[patelatharva]

GeoLiteCity-latest.zip downloaded by script extracts as a directory named: GeoLiteCity_20131203 with a file GeoLiteCity-Blocks.csv inside it. Is this the expected behaviour of updatedb.js script?

[bluesmoon]

There should also be GeoLiteCity-Location.csv

[Tarang]

Its very difficult to trace through the code I tried to check all logs and stuff. What I think was going on was the exit message from the unzip was being intercepted by npm and the installation process was killed.

Sure enough I think this might be a bug in npm itself, after much bruting including trying with the latest versions of unzip the solution is to install the latest version of npm and try again

sudo npm install npm -g

[bluesmoon]

Thanks. If this solves the problem, can you close this issue?

To trace the code, you might have to add console.log statements inside scripts/updatedb.js in the extract() function.

[patelatharva]

It's killed even when the script is run through node command so I guess the older version of Node might be causing the issue. Which version of Node are you running @bluesmoon ? Mine is 0.8.22, yours must be 0.10.x I guess.

[bluesmoon]

Yes, you need to upgrade to 0.10.x. All older versions have a security issue.

[Tarang]

@bluesmoon I tried adding code and a try catch in there too to see if there was any error but its no no avail there is 0 output and the node logs report that the script is terminated externally.

@soichih can close the issue as I didn't initially create it

[bluesmoon]

@Tarangp ok, try with the latest node & npm and open a new issue if it still persists.

[GUI]

I added this note to #45, but just to update this thread for anyone that's interested: I believe this was fixed in master by #36. But since that fix isn't present in v1.1.2 that's on npm, I think that's what's causing this issue. The next version, or running from master should fix this.

[edbond]

reopen or fix, don't close unsolved issue.
node v0.10.33
npm 2.1.10

npm run-script updatedb

> [email protected] updatedb /home/<user>/node-geoip
> node scripts/updatedb.js

Retrieving GeoIPCountryCSV.zip ... DONE
Extracting GeoIPCountryCSV.zip ...
events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: invalid signature: 0x6d74683c (at position: 0x0)
    at /home/<user>/node-geoip/node_modules/unzip/lib/parse.js:56:13
    at /home/<user>/node-geoip/node_modules/unzip/node_modules/pullstream/pullstream.js:101:11
    at process._tickCallback (node.js:419:13)

npm ERR! Linux 2.6.32-220.23.1.el6.x86_64
npm ERR! argv "node" "/usr/bin/npm" "run-script" "updatedb"
npm ERR! node v0.10.33
npm ERR! npm  v2.1.10
npm ERR! code ELIFECYCLE
npm ERR! [email protected] updatedb: `node scripts/updatedb.js`
npm ERR! Exit status 8
npm ERR! 
npm ERR! Failed at the [email protected] updatedb script 'node scripts/updatedb.js'.
npm ERR! This is most likely a problem with the geoip-lite package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node scripts/updatedb.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls geoip-lite
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/<user>/node-geoip/npm-debug.log

cat tmp/GeoIPCountryCSV.zip

<body>Java file has been blocked by your firewall</body></html>

npm-debug.log

0 info it worked if it ends with ok
1 verbose cli [ 'node', '/usr/bin/npm', 'run-script', 'updatedb' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'preupdatedb', 'updatedb', 'postupdatedb' ]
5 info preupdatedb [email protected]
6 info updatedb [email protected]
7 verbose unsafe-perm in lifecycle true
8 info [email protected] Failed to exec updatedb script
9 verbose stack Error: [email protected] updatedb: `node scripts/updatedb.js`
9 verbose stack Exit status 8
9 verbose stack     at EventEmitter.<anonymous> (/usr/lib/node_modules/npm/lib/utils/lifecycle.js:212:16)
9 verbose stack     at EventEmitter.emit (events.js:98:17)
9 verbose stack     at ChildProcess.<anonymous> (/usr/lib/node_modules/npm/lib/utils/spawn.js:14:12)
9 verbose stack     at ChildProcess.emit (events.js:98:17)
9 verbose stack     at maybeClose (child_process.js:756:16)
9 verbose stack     at Process.ChildProcess._handle.onexit (child_process.js:823:5)
10 verbose pkgid [email protected]
11 verbose cwd /home/<user>/node-geoip
12 error Linux 2.6.32-220.23.1.el6.x86_64
13 error argv "node" "/usr/bin/npm" "run-script" "updatedb"
14 error node v0.10.33
15 error npm  v2.1.10
16 error code ELIFECYCLE
17 error [email protected] updatedb: `node scripts/updatedb.js`
17 error Exit status 8
18 error Failed at the [email protected] updatedb script 'node scripts/updatedb.js'.
18 error This is most likely a problem with the geoip-lite package,
18 error not with npm itself.
18 error Tell the author that this fails on your system:
18 error     node scripts/updatedb.js
18 error You can get their info via:
18 error     npm owner ls geoip-lite
18 error There is likely additional logging output above.
19 verbose exit [ 1, true ]

[bluesmoon]

reopening, feel free to submit a patch.

[echiu64]

@edbond looks like you have a firewall that is preventing you from downloading GeoIPCountryCSV.zip based on the output you provided. I'd suggest you fix that first, retry and then see if you're still having issues?

[edbond]

@echiu64 There is no firewall afaik, I am able to download files mentioned in update script using wget.

[bluesmoon]

can you use a packet capture tool like wireshark to see if there's any difference between the wget request and the one that goes out from node?

[edbond]

tcpdump except for npm install:

GET /download/geoip/database/GeoIPCountryCSV.zip HTTP/1.1
Host: geolite.maxmind.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.36 Safari/537.36
Connection: keep-alive

HTTP/1.1 200 OK
Server: Netscreen 100
Connection: close
Content-Length: 142
Conent-Type: text/html

<html><head><title>Java Blocking</title></head>^M<body>Java file has been blocked by your firewall</body></html>

and tcpdump for wget:

GET /download/geoip/database/GeoIPCountryCSV.zip HTTP/1.0
User-Agent: Wget/1.12 (linux-gnu)
Accept: */*
Host: geolite.maxmind.com
Connection: Keep-Alive

HTTP/1.1 200 OK
Server: Netscreen 100
Connection: close
Content-Length: 142
Conent-Type: text/html

<html><head><title>Java Blocking</title></head>^M<body>Java file has been blocked by your firewall</body></html>

GET /download/geoip/database/GeoIPCountryCSV.zip HTTP/1.0
Range: bytes=113-
User-Agent: Wget/1.12 (linux-gnu)
Accept: */*
Host: geolite.maxmind.com
Connection: Keep-Alive

HTTP/1.1 206 Partial Content
Date: Tue, 02 Dec 2014 16:35:22 GMT
Content-Type: application/zip
Content-Length: 1584297
Connection: keep-alive
Set-Cookie: __cfduid=de1e9e26fd1f0ef3cdd306386608045411417538122; expires=Wed, 02-Dec-15 16:35:22 GMT; path=/; domain=.maxmind.com; HttpOnly
Last-Modified: Wed, 05 Nov 2014 19:25:33 GMT
CF-Cache-Status: HIT
Expires: Tue, 02 Dec 2014 20:35:22 GMT
Cache-Control: public, max-age=14400
Content-Range: bytes 113-1584409/1584410
Server: cloudflare-nginx
CF-RAY: 1928efeecd220773-EWR

Here is the output of wget, maxmind does return the same 'blocked' message and wget retry with Range header:

wget http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
--2014-12-02 11:36:10--  http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
Resolving geolite.maxmind.com... 141.101.115.190, 141.101.114.190, 2400:cb00:2048:1::8d65:72be, ...
Connecting to geolite.maxmind.com|141.101.115.190|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 142
Saving to: “GeoIPCountryCSV.zip”

79% [================================================================================>                      ] 113         --.-K/s   in 0s      

2014-12-02 11:36:10 (2,18 MB/s) - Connection closed at byte 113. Retrying.

--2014-12-02 11:36:11--  (try: 2)  http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
Connecting to geolite.maxmind.com|141.101.115.190|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 1584410 (1,5M), 1584297 (1,5M) remaining [application/zip]
Saving to: “GeoIPCountryCSV.zip”

100%[======================================================================================================>] 1 584 410   7,81M/s   in 0,2s    

2014-12-02 11:36:12 (7,81 MB/s) - “GeoIPCountryCSV.zip” saved [1584410/1584410]

Note that npm install works on my local machine fine, doesn't works on server (CentOS release 6.2 (Final)).

[echiu64]

Note that your response shows "Server: Netscreen 100", which is not from maxmind, it's on your end or your ISP:

Here is what it should look like, note the Server header is cloudflare-nginx:

$ curl -I http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
HTTP/1.1 200 OK
Date: Wed, 03 Dec 2014 00:37:42 GMT
Content-Type: application/zip
Content-Length: 1537529
Connection: keep-alive
Set-Cookie: __cfduid=d75eec9365b869c53fe1097a127ae205e1417567062; expires=Thu, 03-Dec-15 00:37:42 GMT; path=/; domain=.maxmind.coy
Last-Modified: Tue, 02 Dec 2014 21:43:42 GMT
CF-Cache-Status: HIT
Expires: Wed, 03 Dec 2014 04:37:42 GMT
Cache-Control: public, max-age=14400
Accept-Ranges: bytes
Server: cloudflare-nginx
CF-RAY: 192bb27dc90509b2-ORD

[edbond]

@echiu64 the Netscreen response is from maxmind cloudflare CDN too, here is the dump of that response:

11:35:30.626509 IP (tos 0x0, ttl 54, id 55195, offset 0, flags [DF], proto TCP (6), length 259)
    141.101.115.190.http > <myhost>.50381: Flags [FP.], cksum 0x7303 (correct), seq 1:220, ack 235, win 16, length 219
[email protected].. .P...#.\....P...s...HTTP/1.1 200 OK
Server: Netscreen 100
Connection: close
Content-Length: 142
Conent-Type: text/html

Note source IP 141.101.115.190

→ host geolite.maxmind.com
geolite.maxmind.com is an alias for geolite.maxmind.com.cdn.cloudflare.net.
geolite.maxmind.com.cdn.cloudflare.net has address 141.101.114.190
geolite.maxmind.com.cdn.cloudflare.net has address 141.101.115.190
geolite.maxmind.com.cdn.cloudflare.net has IPv6 address 2400:cb00:2048:1::8d65:72be
geolite.maxmind.com.cdn.cloudflare.net has IPv6 address 2400:cb00:2048:1::8d65:73be

[echiu64]

Try

curl -I http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip

And you will see the difference in headers, since your Netscreen is filtering.

You can try this from a free AWS micro instance if you signup for an AWS account.

You are clearly being filtered by a Netscreen. Your "Server" header shows "Netscreen 100" instead of "cloudflare-nginx". The DNS results for geolite.maxmind.com aren't really relevant, since the Netscreen is a transparent filter/proxy in this case.

[edbond]

Don't see any Netscreen:

node-geoip]$ curl -v -I http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
* About to connect() to geolite.maxmind.com port 80 (#0)
*   Trying 141.101.114.190... connected
* Connected to geolite.maxmind.com (141.101.114.190) port 80 (#0)
> HEAD /download/geoip/database/GeoIPCountryCSV.zip HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2
> Host: geolite.maxmind.com
> Accept: */*
> 
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: Wed, 03 Dec 2014 05:40:31 GMT
Date: Wed, 03 Dec 2014 05:40:31 GMT
< Content-Type: application/zip
Content-Type: application/zip
< Content-Length: 1537529
Content-Length: 1537529
< Connection: keep-alive
Connection: keep-alive
< Set-Cookie: __cfduid=d6877606014c36517c2e6e830c4db15c41417585231; expires=Thu, 03-Dec-15 05:40:31 GMT; path=/; domain=.maxmind.com; HttpOnly
Set-Cookie: __cfduid=d6877606014c36517c2e6e830c4db15c41417585231; expires=Thu, 03-Dec-15 05:40:31 GMT; path=/; domain=.maxmind.com; HttpOnly
< Last-Modified: Tue, 02 Dec 2014 21:43:42 GMT
Last-Modified: Tue, 02 Dec 2014 21:43:42 GMT
< CF-Cache-Status: HIT
CF-Cache-Status: HIT
< Expires: Wed, 03 Dec 2014 09:40:31 GMT
Expires: Wed, 03 Dec 2014 09:40:31 GMT
< Cache-Control: public, max-age=14400
Cache-Control: public, max-age=14400
< Accept-Ranges: bytes
Accept-Ranges: bytes
< Server: cloudflare-nginx
Server: cloudflare-nginx
< CF-RAY: 192d6e1373e100ae-EWR
CF-RAY: 192d6e1373e100ae-EWR

< 
* Connection #0 to host geolite.maxmind.com left intact
* Closing connection #0

[echiu64]

That looks normal now, you have a right Content-Length. What happens when you run the update now?

It could also be your Netscreen allows the HEAD request through, but the GET will fail as it doesn't like the content. HEAD doesn't retrieve any content, so the filter rules may not apply.

[edbond]

It still doesn't work. I think that the point of this issue is to improve user experience installing geoip not searching how Netscreen rules affect traffic.

I suggest

• retry two times downloading zip/gz in case unzip failed
• show a warning to user that downloading failed and instructions how to download them manually like

[warn] GeoIP databases download failed, try to download them manually using following commands and run 'npm install' again
mkdir ~/.node-geoip && cd ~/.node-geoip
wget/curl <list-of-urls>
npm install ....

• search for downloaded zip/gz in ~/.node-geoip first and download if not found

[echiu64]

Fair enough. To that end, they should also ensure that they don't have any firewall or antivirus blocking or impacting the download. As well as provide the contents of relevant files. I think the cat output of GeoIPCountryCSV.zip was the most telling of all. It was HTML output from Netscreen vs an actual zip file.

I think this issue can be closed, this really isn't a geoip issue. If you want to update the README.md, feel free to send a pull request with your suggestions and I'm sure the maintainer will accept it.

[bluesmoon]

Looking for a pre-downloaded file isn't the solution because the file might go out of date, and node-geoip will try to download a new one at some point if you've configured it to do so.

[edbond]

@bluesmoon How about storing md5/sha1 of latest version and compare against files? Warn user and try to download newest version.

[bluesmoon]

How would we know the md5/sha1 of the latest version when we need to download it first to find out? Whatever I put into the package will be out of date in 30 days, and I don't plan on releasing a new package every 30 days just to update the checksum.

[Alex777UAE]

Same problem exists on Mac OS 10.11 El Capitan, MacBook Pro (Retina, 13-inch, Early 2015), 8GB RAM.
I suspect its because of old unzip.

[email protected] updatedb /Users/alex/Projects/ap/node_modules/geoip-lite
node scripts/updatedb.js

Retrieving GeoIPCountryCSV.zip ... DONE
Extracting GeoIPCountryCSV.zip ... DONE
Processing Data (may take a moment) ... DONE

Retrieving GeoIPv6.csv ... DONE
Processing Data (may take a moment) ... DONE

Retrieving GeoLiteCity-latest.zip ... DONE
Extracting GeoLiteCity-latest.zip ...

and process getting killed. playing with ulimits (and root permissions) doesn't make a difference.

in tmp/GeoLiteCity_20150901/
there is only GeoLiteCity-Blocks.csv file, fully unpacked. 2nd file is missing. same as for @Tarang and @patelatharva