mteodoro / mmutils Goto Github PK

Hi there!
Maybe i'll lucky and someone can help me with this deprecated project.
I try to build geolitecity from csv (Source: GeoLiteCity-flat-sorted.csv.zip) with csv2dat.py -w GeoLiteCity.dat mmcity GeoLiteCity-flat-sorted.csv. When i done, i check it with, for example, ip 191.243.53.129 .
In debug file (when we use csv2dat.py with -d key) i see:

2148767 ["14702888 (('BR', 'Parana', 'Ampere', '', '-25.9167', '-53.4667', '', '20'), IPv4Network('191.243.48.0/22'))", "17135708 (('BR', 'Santa Catarina', 'Abelardo Luz', '89830', '-26.5983', '-52.2232', '', '50'), IPv4Network('191.243.52.0/22'))"]
2148768 ["16892194 (('BR', 'Minas Gerais', 'Pompeu', '35640', '-19.1495', '-44.9149', '', '10'), IPv4Network('191.243.56.0/22'))", 2148769]

BUT when i ask my GeoLiteCity.dat file "tell me, who is 191.243.53.129" with python (pygeoip, GeoIP modules) or geoiplookup i got: US, Pennsylvania, N/A, Philadelphia, 19123, 39.964298, -75.144402, 504, 500
So, maybe someone can explain me, why it done? Why info in .dat file is wrong? And how fix it?
P.S. I'm desperate

can you set a license for the scripts? in doubt i would recommend MIT.

Traceback (most recent call last):
  File "./csv2dat.py", line 475, in <module>
    rval = main()
  File "./csv2dat.py", line 471, in main
    return cmd(opts, args)
  File "./csv2dat.py", line 439, in build_dat
    r.load(opts, args)
  File "./csv2dat.py", line 194, in load
    for nets, data in self.gen_nets(opts, args):
  File "./csv2dat.py", line 317, in gen_nets
    lo, hi = row[:2]
ValueError: need more than 0 values to unpack

`Please help on this error.

It would be nice to restructure package that way it could be installed via setup tools and used as standalone library as the standalone tool it does at the moment.
We could make PR for this, if maintainer interested in it.

Quick question - Is this error related to my environment, the import file, or something else? This file does not contain any IPv6 addresses...

root# ./csv2dat.py -w Citrix_mynewdb.dat mmcity myoldcsv.csv
Traceback (most recent call last):
File "./csv2dat.py", line 270, in
class ASNv6RadixTree(ASNRadixTree):
File "./csv2dat.py", line 274, in ASNv6RadixTree
edition = pygeoip.const.ASNUM_EDITION_V6
AttributeError: 'module' object has no attribute 'ASNUM_EDITION_V6'
[root]#

/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 /Users/admin/PycharmProjects/geoip/csv2dat.py -w GeoIPCity.dat mmorg GeoIPCity.csv
Traceback (most recent call last):
File "/Users/admin/PycharmProjects/geoip/csv2dat.py", line 475, in
rval = main()
File "/Users/admin/PycharmProjects/geoip/csv2dat.py", line 471, in main
return cmd(opts, args)
File "/Users/admin/PycharmProjects/geoip/csv2dat.py", line 439, in build_dat
r.load(opts, args)
File "/Users/admin/PycharmProjects/geoip/csv2dat.py", line 194, in load
for nets, data in self.gen_nets(opts, args):
File "/Users/admin/PycharmProjects/geoip/csv2dat.py", line 261, in gen_nets
for lo, hi, asn in gen_csv(fileinput.input(args)):
ValueError: too many values to unpack

I need one, thank you.

Hi

My csv file contains characters (isp names) out range of ascii.
After transforming, my application could read the dat file, but result in unreadable code.

Thank you for help.

I'm really looking to use this utility to modify info from GeoIP Country (GEO-108) and GeoIP ISP (GEO-124) csv files and produce the corresponding dat files. For the country we only access the .country_code_by_addr and for the isp we only access the .org_by_addr methods of the python api.

Please, any info or help would be appreciated. Thank you for making this utility available in the first place.

Hi there

I'd first like to thank you for your csv2dat.py script. I have used it to merge the lat/long positions of our internal 10/8 network subnets into the maxmind CSV files and now we can do things like have maps showing our internal devices using websites/etc as well as Internet :-)

However, now we're seeing applications that only support the newer MMDB format. Have you thought about adding support for that as an option? So far there only seems to be a perl "writer" from Maxmind - but nothing for python :-(

In any case, thanks for your work!

Jason

Hi there,

I tried to use this project to convert GeoIP2 csv to dat file. But it did not work. Do you have plan to add this feature ?

I've run the conversion a few times, and I find that the script doesn't handle when there is duplicate IPs.

Line 205 gets an except when there is a duplicate start IP address

• Don't ask why, I gotta yell at someone for this..

Line 189 gets an except if you define overlapping IP address

• I'm over loading the Maxmind database with my own data, in some cases, this is public IP space that i'm providing more granular data.

c:\Temp\mmdb-convert-master>mmdb-convert.py GeoLite2-City.mmdb > GeoIPCity.dat
Traceback (most recent call last):
File "C:\Temp\mmdb-convert-master\mmdb-convert.py", line 463, in
metadata, the_tree, _ = parse_mm_file(content)
File "C:\Temp\mmdb-convert-master\mmdb-convert.py", line 327, in parse_mm_file
tree = parse_search_tree(s[:tree_size], mm['record_size'].int_val())
File "C:\Temp\mmdb-convert-master\mmdb-convert.py", line 123, in parse_search_tree
left, right = to_leftright(s[p:p+record_bytes])
File "C:\Temp\mmdb-convert-master\mmdb-convert.py", line 80, in to_int28
a, b = unpack("!LL", s + b'\x00')
NameError: name 'unpack' is not defined

Hi there

I want to use csv2dat.py to merge into GeoIP geographic locations for our 10/8 network (200 sites, I've got lat/long/city by subnet)

So I duplicated the CSV format of GeoLiteCity-Location.csv/GeoLiteCity-Blocks.csv appended my data - but csv2dat crashes as follows when I run it. BTW, if I run it on the original GeoIP csv files, it works fine, and I even made a 10-line version where I hand-picked some data points from GeoIP plus mine and that worked too. I suspect there's a simple bad char or something in my much larger data set - but it's too big to parse by eyeball. How could I add some debugging code to at least tell me what line in locations/blocks is causing the fault?

Here's the runtime error. Thanks!

csv2geoipDAT.py -w mmcity.dat -l locations.csv mmcity blocks.csv
Traceback (most recent call last):
File "/usr/local/bin/csv2geoipDAT.py", line 475, in
rval = main()
File "/usr/local/bin/csv2geoipDAT.py", line 471, in main
return cmd(opts, args)
File "/usr/local/bin/csv2geoipDAT.py", line 439, in build_dat
r.load(opts, args)
File "/usr/local/bin/csv2geoipDAT.py", line 196, in load
self[net] = data
File "/usr/local/bin/csv2geoipDAT.py", line 170, in setitem
if not node.lhs:
AttributeError: 'tuple' object has no attribute 'lhs'