Coder Social home page Coder Social logo

manuf's Introduction

manuf

Build Status Build Status

Parser library for Wireshark's OUI database.

Converts MAC addresses into a manufacturer using Wireshark's OUI database.

Optimized for quick lookup performance by reading the entire file into memory on initialization. Maps ranges of MAC addresses to manufacturers and comments (descriptions). Contains full support for netmasks and other strange things in the database.

See Wireshark's OUI lookup tool.

Written by Michael Huang (coolbho3k).

Install

With PyPi

pip install manuf

Or Manually

git clone https://github.com/coolbho3k/manuf
cd manuf
python setup.py install

Usage

As a library:

>>> from manuf import manuf
>>> p = manuf.MacParser(update=True)
>>> p.get_all('BC:EE:7B:00:00:00')
Vendor(manuf='AsustekC', comment='ASUSTek COMPUTER INC.')
>>> p.get_manuf('BC:EE:7B:00:00:00')
'AsustekC'
>>> p.get_comment('BC:EE:7B:00:00:00')
'ASUSTek COMPUTER INC.'

As a command line:

$ manuf BC:EE:7B:00:00:00
Vendor(manuf='AsustekC', comment='ASUSTek COMPUTER INC.')

Use a manuf file in a custom location:

$ manuf --manuf ~/manuf BC:EE:7B:00:00:00
Vendor(manuf='AsustekC', comment='ASUSTek COMPUTER INC.')

Automatically update the manuf file from Wireshark's git:

$ manuf --update BC:EE:7B:00:00:00
Vendor(manuf='AsustekC', comment='ASUSTek COMPUTER INC.')

Note, that this command will update the manuf file bundled with this package. If you do not wish to modify this, or do not have permissions to do so, you must specify a custom manuf file to perform an update.

$ manuf --update --manuf ~/manuf BC:EE:7B:00:00:00
Vendor(manuf='AsustekC', comment='ASUSTek COMPUTER INC.')

Alternatively you can call the program with:

python -m manuf

or by executing the manuf.py script directly

./manuf/manuf.py # From the install folder

Features and advantages of manuf

Note: the examples use the manuf file provided in the first commit, 9a180b5.

manuf.py is more accurate than more naive scripts that parse the manuf file. Critically, it contains support for netmasks.

For a usual entry, such as BC:EE:7B (AsustekC), the manufacturer "owns" the last half (24 bits) of the MAC address and is free to assign the addresses BC:EE:7B:00:00:00 through BC:EE:7B:FF:FF:FF, inclusive, to its devices.

However, entries like the following also appear commonly in the file:

00:1B:C5:00:00:00/36	Convergi               # Converging Systems Inc.
00:1B:C5:00:10:00/36	OpenrbCo               # OpenRB.com, Direct SIA

/36 is a netmask, which means that the listed manufacturer "owns" only the last 12 bits of the MAC address instead of the usual 24 bits (since MAC addresses are 48 bits long, and 48 bits - 36 bits = 12 bits).

This means that Converging Systems is only free to assign the addresss block 00:1B:C5:00:00:00 through 00:1B:C5:00:0F:FF. Anything after that belongs to other manufacturers. manuf.py takes this fact into account:

>>> p.get_manuf('00:1B:C5:00:00:00')
'Convergi'
>>> p.get_manuf('00:1B:C5:00:0F:FF')
'Convergi'
>>> p.get_manuf('00:1B:C5:00:10:00')
'OpenrbCo'

Even Wireshark's web lookup tool fails here. "00:1B:C5:00:0F:FF" returns only "IEEE REGISTRATION AUTHORITY" while it should instead return "Converging Systems Inc." If a netmask is not explicitly specified, a netmask of /24 is implied. Since this covers most of the entries, most tools only parse the first 24 bits.

manuf.py fully supports even more esoteric entries in the database. For example, consider these two entries:

01-80-C2-00-00-30/45	OAM-Multicast-DA-Class-1
01-80-C2-00-00-38/45	OAM-Multicast-DA-Class-2

With a netmask of /45, only the last 3 bits of the address are significant. This means that a device is considered "OAM-Multicast-DA-Class-1" only if the last digit falls between 0x0 and 0x7 and "OAM-Multicast-DA-Class-2" only if the last digit falls between 0x8 and 0xF.

If the last octet is 0x40 or over, or 0x2F or under, the address doesn't belong to any manufacturer.

>>> p.get_manuf('01:80:C2:00:00:2F')
>>> p.get_manuf('01:80:C2:00:00:30')
'OAM-Multicast-DA-Class-1'
>>> p.get_manuf('01:80:C2:00:00:37')
'OAM-Multicast-DA-Class-1'
>>> p.get_manuf('01:80:C2:00:00:38')
'OAM-Multicast-DA-Class-2'
>>> p.get_manuf('01:80:C2:00:00:3F')
'OAM-Multicast-DA-Class-2'
>>> p.get_manuf('01:80:C2:00:00:40')

Again, the official lookup tool fails here as well, with "01:80:C2:00:00:31" returning no results.

Algorithm

Although optimized for correctness, manuf.py is also quite fast, with average O(1) lookup time, O(n) setup time, and O(n) memory footprint.

First, the entire manuf file is read into memory. Each manuf line is stored in a dict mapping a tuple calculated from the MAC address and netmask to each manuf:

((48 - netmask), macaddress >> (48 - netmask))

The (48 - netmask) value is called the "bits left" value in the code.

For example, Converging Systems' MAC is 0x001BC5000000 and its netmask is 36, so its key in the dict is this:

(12, 0x001BC5000000 >> 12)

To lookup "00:1B:C5:00:0F:FF" we will check the dict beginning with a "bits left" value of 0, incrementing until we find a match or go over 47 (which means we have no match):

(0, 0x001BC5000FFF >> 0)
(1, 0x001BC5000FFF >> 1)
(2, 0x001BC5000FFF >> 2)
...
(12, 0x001BC5000FFF >> 12)

Since (12, 0x001BC5000FFF >> 12) equals (12, 0x001BC5000000 >> 12), we have a match on the 13th iteration of the loop.

Copying

This library does not link to Wireshark's manuf database; it merely parses it, so I have chosen to publish it under the LGPLv3 and Apache License 2.0 instead of the GPLv2. The manuf database is provided for your convenience in this repository, but will not be updated.

  • License for Python library: LGPLv3 and Apache License 2.0 (dual licensed)
  • License for manuf database: GPLv2

The latest version of the manuf database can be found in the Wireshark git repository. The database there is updated about once a week, so you may want to grab the latest version to use instead of using the one provided here by using the --update flag on the command line:

manuf --update

Run tests

python -m unittest manuf.test.test_manuf

manuf's People

Contributors

abngal avatar antonandreyev avatar beercow avatar computeallthethings avatar coolbho3k avatar gte620v avatar jcannell avatar neil-orans avatar rene-d avatar rlaager avatar tirkarthi avatar tristanlatr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

manuf's Issues

Error 503

$ manuf -u
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/manuf/manuf.py", line 164, in update
    response = urlopen(Request(wfa_url, headers={'User-Agent': 'Mozilla'}))
  File "/usr/lib/python3.9/urllib/request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 503: Service Unavailable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/manuf", line 33, in <module>
    sys.exit(load_entry_point('manuf==1.1.1', 'console_scripts', 'manuf')())
  File "/usr/lib/python3.9/site-packages/manuf/manuf.py", line 324, in main
    parser = MacParser(manuf_name=args.manuf, update=args.update)
  File "/usr/lib/python3.9/site-packages/manuf/manuf.py", line 70, in __init__
    self.update()
  File "/usr/lib/python3.9/site-packages/manuf/manuf.py", line 166, in update
    raise URLError("Failed downloading WFA database")
urllib.error.URLError: <urlopen error Failed downloading WFA database>

It seems https://code.wireshark.org/review/gitweb?p=wireshark.git;a=blob_plain;f=manuf has moved to https://gitlab.com/wireshark/wireshark/raw/master/manuf

Please update the script. Thanks

Deprecating Python2

Helo everyone,

Python2 is deprecated and keeping it supported prevent us to have technicals improvement.

I suggest the next release don't support Python2,

Please write your thoughts,

Tristan

run library module as a script

It would be nice if you could run an update like so

python -m manuf --update

Instead of having to find the path to the python script
Adding a __main__.py allows this


from manuf import main

if __name__ == "__main__":
    main()

Strange import path

Hey,
It seems like current version published on pip (and master here as well) exposes MacParser class in manuf.manuf.MacParser path. This is pretty strange, and makes an example in README slightly wrong (import manuf ; p = manuf.MacParser())

You can either from .manuf import MacParser in manuf/__init__.py as a backward-compatible solution, or just get rid of manuf.py all together and move everything over to __init__.py.

Well Known Addresses in separate file

Need a way to download and merge manuf and wfa files.

Wireshark commit:

Put the well-known addresses into a separate file from OUIs

Having two distinct logical concepts (OUI and Well Known Address)
concatenated to a single "manuf" file is needlessly obfuscating
the WKA feature.

Have a distinct "wka" file instead and just skip the cat.

Change-Id: I46f53b0015a37331d65f8cfac7cbbd499dd0c5b8
Reviewed-on: https://code.wireshark.org/review/22742
Petri-Dish: Michael Mann [email protected]
Tested-by: Petri Dish Buildbot [email protected]
Reviewed-by: Michael Mann [email protected]

Planning to drop python2 support

Hello, just a quick note to advise that I plan to drop support python2 in the next version.
If you need manuf to still supports python2, please send PRs to setup proper testing environment for python2 and fix eventual syntax error.

SyntaxWarning


$ manuf -h
/usr/lib/python3.8/site-packages/manuf/manuf.py:147: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if response.code is 200:
/usr/lib/python3.8/site-packages/manuf/manuf.py:167: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if response.code is 200:

From daniel-leicht#4

Failed to connect - Has URL permission changed to OUI Database?

A working program, which I've not amended now fails. Has permission to the OUI database changed to deny access?

mac_addr_test.py", line 37, in
p = manuf.MacParser(update=True)
manuf.py", line 70, in init
self.update()
manuf.py", line 146, in update
raise URLError("Failed downloading OUI database")
urllib.error.URLError:

Wireshark removed the manuf file

Looks like the wireshark project has replaced the manuf file with a .c file containing static arrays.

I'll try to come up with a pull request to fix this. It might take me a while.

Failing to acquire database when update=?True

Just in the last few days I noticed my scripts using manuf.py started failing to get the manuf database from gitlab.

I'm not sure what changed, but I tried installing it on my vdi clean and it is reproducible.

I am able to see the database in the manuf_url from the module using chrome on the same system.

Nick

$ python3
Python 3.6.8 (default, Feb 14 2019, 22:09:48)
[GCC 7.4.0] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from manuf import manuf
>>> p = manuf.MacParser(update=True)
Traceback (most recent call last):
  File "/home/nicko/manuf/manuf/manuf.py", line 142, in update
    response = urlopen(manuf_url)
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 564, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 756, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nicko/manuf/manuf/manuf.py", line 68, in __init__
    self.update()
  File "/home/nicko/manuf/manuf/manuf.py", line 144, in update
    raise URLError("Failed downloading OUI database")
urllib.error.URLError: <urlopen error Failed downloading OUI database>
>>>

Publish into pip

Tried to install with:

$ pip install manuf
Downloading/unpacking manuf
  Could not find any downloads that satisfy the requirement manuf
Cleaning up...
No distributions at all found for manuf

New release on PyPI

Hi,

thanks for this tool! After the manuf file format change, would it be possible to get a new release with the new parser published to PyPI?

Wireshark file format change

The formatting of the wireshark.org file appears to have changed. Previously it used “ # “ as a separator for the full manufacturer name and a comment.

The file now appears to use “ # “ solely as a comment and as a result, after you update the manuf file from Wireshark's git, only true comments are displayed.

python manuf.py 08:00:89
Vendor(manuf=u'Kinetics', comment=u'Kinetics\t\t\tAppleTalk-Ethernet interface')

python manuf.py 1C:BD:B9:FF:E9:18
Vendor(manuf=u'D-LinkIn', comment=u'D-Link International')

python manuf.py D8:FC:93
Vendor(manuf=u'IntelCor', comment=u'Intel Corporate')

python manuf.py 00:40:1C
Vendor(manuf=u'AstPenti', comment=u'AST\t\t\t\tPentium/90 PC (emulating AMD EISA card)')

After update

python manuf.py --update
python manuf.py 08:00:89
Vendor(manuf=u'Kinetics', comment=u'AppleTalk-Ethernet interface')
python manuf.py 1C:BD:B9:FF:E9:18
Vendor(manuf=u'D-LinkIn', comment=None)
python manuf.py D8:FC:93
Vendor(manuf=u'IntelCor', comment=None)
python manuf.py 00:40:1C
Vendor(manuf=u'AstPenti', comment=u'Pentium/90 PC (emulating AMD EISA card)')

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.