mike01 / pypacker Goto Github PK

Method pypacker.hexdump raise exception on bytes object:
File "/home/user/pyve/pcap-tools3/lib/python3.3/site-packages/pypacker/pypacker.py", line 1046, in hexdump
line = re.sub(PROG_VISIBLE_CHARS, b".", line)
File "/home/user/pyve/pcap-tools3/lib/python3.3/re.py", line 170, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: can't use a string pattern on a bytes-like object

As the title suggests,

I'm trying to use the TCP class in layer 4 to extract TCP header information but I don't know what method to use to get the number of bytes.

Wireshark will give the following information from a single packet:

No. Time Source Destination Protocol Length Info
1 0.000000 192.168.1.164 142.104.5.64 TCP 62 1200→80 [SYN] Seq=0 Win=16384 Len=0 MSS=1460 SACK_PERM=1

Frame 1: 62 bytes on wire (496 bits), 62 bytes captured (496 bits)
Ethernet II, Src: Intel_a1:48:60 (00:0e:35:a1:48:60), Dst: 3com_20:98:c5 (00:10:5a:20:98:c5)
Internet Protocol Version 4, Src: 192.168.1.164 (192.168.1.164), Dst: 142.104.5.64 (142.104.5.64)
Transmission Control Protocol, Src Port: 1200 (1200), Dst Port: 80 (80), Seq: 0, Len: 0

What I'm looking for is the Length given from the TCP header or the bytes captured from the frame.

This is a great module and thanks in advance.

Any chance you'd consider making this library python2 compatible for those of us stuck on python2?

----> 1 from pypacker import ppcap
2 from pypacker.layer12 import ethernet
3 from pypacker.layer3 import ip
4 from pypacker.layer4 import tcp
5

ImportError: cannot import name ppcap

Came across an error when messing with the dns module. For some reason, it seems to trip on certain packets and throw a struct.error.
struct.error: unpack requires a bytes object of length 442

I'm running it on the smbtorture.cap file here

For the full context, here is the code I'm running, or trying to run. It works fine on other pcap files.
(sorry for the lack of comments, this was just a quick and dirty script)

import pcapy
from pypacker.pypacker import dns_name_decode
from pypacker.layer12 import ethernet
from pypacker.layer567 import dns
import os
from socket import inet_ntop, AF_INET, AF_INET6

# https://wiki.wireshark.org/SampleCaptures#General_.2F_Unsorted
#PCAPFILE = os.path.join(os.environ['HOME'], "pcap", "SkypeIRC.cap")
# https://wiki.wireshark.org/SampleCaptures#Server_Message_Block_.28SMB.29.2FCommon_Internet_File_System_.28CIFS.29
PCAPFILE = os.path.join(os.environ['HOME'], "pcap", "smbtorture.cap")
BPF = "port 53"

capture = pcapy.open_offline(PCAPFILE)
capture.setfilter(BPF)

while True:
    try:
        header, packet = capture.next()
    except pcapy.PcapError:
        break

    pkt = ethernet.Ethernet(packet)
    dnsP = pkt.highest_layer
    if not isinstance(dnsP, dns.DNS):
        continue

    print("{0:016b}".format(dnsP.flags))
    qr_flag = dnsP.flags >> 15
    if qr_flag == dns.DNS_Q:
        for query in dnsP.queries:
            print(dnsP.id, "query", query.name_s)
    else: #if qr_flag == dns.DNS_A:
        if not dnsP.answers:
            print(dnsP.id, "response", None)
            continue
        for answer in dnsP.answers:
            if answer.type == dns.DNS_A:
                print(dnsP.id, "response", "A", inet_ntop(AF_INET, answer.address))
            elif answer.type == dns.DNS_AAAA:
                print(dnsP.id, "response", "AAAA", inet_ntop(AF_INET6, answer.address))
            elif answer.type == dns.DNS_CNAME:
                # decoding name doesn't always work if it has \xc0 pointers
                print(dnsP.id, "response", "CNAME", repr(dns_name_decode(answer.address)))
            elif answer.type == dns.DNS_PTR:
                print(dnsP.id, "response", "PTR", repr(dns_name_decode(answer.address)))
            else:
                print(dnsP.id, "response", "OTHER", repr(answer.address))

The full output, including the exception traceback:

> python3 ~/Python/quick_dns_test.py
0000000100000000
22185 query time.windows.com.
1000000110000000
22185 response None
0000000100000000
53928 query time.windows.com.localdomain.
1000000110000011
53928 response None
0000000100000000
427 query time.windows.com.
Traceback (most recent call last):
  File "/home/user/Python/quick_dns_test.py", line 30, in <module>
    print("{0:016b}".format(dnsP.flags))
  File "/home/user/python3/lib/python3.4/site-packages/pypacker/pypacker.py", line 135, in getfield_simple
    obj._unpack()
  File "/home/user/python3/lib/python3.4/site-packages/pypacker/pypacker.py", line 776, in _unpack
    header_unpacked = self._header_format.unpack(self._header_cached)
struct.error: unpack requires a bytes object of length 442

And for even more completion, here is the tcpdump output, suggesting to me that at least the packets aren't completely malformed:

> tcpdump -nnr ~/pcap/smbtorture.cap port 53
reading from file /home/user/pcap/smbtorture.cap, link-type EN10MB (Ethernet)
23:38:26.418664 IP 192.168.114.129.49157 > 192.168.114.1.53: 22185+ AAAA? time.windows.com. (34)
23:38:27.261990 IP 192.168.114.1.53 > 192.168.114.129.49157: 22185 0/1/0 (102)
23:38:27.268868 IP 192.168.114.129.49157 > 192.168.114.1.53: 53928+ AAAA? time.windows.com.localdomain. (46)
23:38:27.471756 IP 192.168.114.1.53 > 192.168.114.129.49157: 53928 NXDomain 0/1/0 (121)
23:38:27.563782 IP 192.168.114.129.49157 > 192.168.114.1.53: 427+ A? time.windows.com. (34)
23:38:27.743738 IP 192.168.114.1.53 > 192.168.114.129.49157: 427 1/5/5 A 207.46.130.100 (228)
23:38:28.844217 IP 192.168.114.129.49157 > 192.168.114.1.53: 55979+ A? teredo.ipv6.microsoft.com. (43)
23:38:28.844590 IP 192.168.114.1.53 > 192.168.114.129.49157: 55979 4/5/5 A 64.4.25.86, A 64.4.25.80, A 64.4.25.82, A 64.4.25.84 (285)
23:38:32.810654 IP 192.168.114.129.49157 > 192.168.114.1.53: 37291+ A? isatap.localdomain. (36)
23:38:32.948270 IP 192.168.114.1.53 > 192.168.114.129.49157: 37291 NXDomain 0/1/0 (111)
23:38:32.976324 IP 192.168.114.129.49157 > 192.168.114.1.53: 35498+ A? isatap.localdomain. (36)
23:38:32.976572 IP 192.168.114.1.53 > 192.168.114.129.49157: 35498 NXDomain 0/1/0 (111)
23:38:39.029400 IP 192.168.114.129.49157 > 192.168.114.1.53: 44972+ A? teredo.ipv6.microsoft.com. (43)
23:38:39.080072 IP 192.168.114.1.53 > 192.168.114.129.49157: 44972 4/5/5 A 64.4.25.84, A 64.4.25.86, A 64.4.25.80, A 64.4.25.82 (285)
23:38:39.083152 IP 192.168.114.129.49157 > 192.168.114.1.53: 11695+ A? isatap.localdomain. (36)
23:38:39.120008 IP 192.168.114.1.53 > 192.168.114.129.49157: 11695 NXDomain 0/1/0 (111)
23:38:39.122330 IP 192.168.114.129.49157 > 192.168.114.1.53: 21422+ A? isatap.localdomain. (36)
23:38:39.155815 IP 192.168.114.1.53 > 192.168.114.129.49157: 21422 NXDomain 0/1/0 (111)
23:38:39.703137 IP 192.168.114.129.49178 > 192.168.114.1.53: 63392+ A? time.windows.com. (34)
23:38:39.703434 IP 192.168.114.1.53 > 192.168.114.129.49178: 63392 1/5/5 A 207.46.130.100 (228)
23:38:39.711962 IP 192.168.114.129.49157 > 192.168.114.1.53: 19873+ A? isatap.localdomain. (36)
23:38:39.712165 IP 192.168.114.1.53 > 192.168.114.129.49157: 19873 NXDomain 0/1/0 (111)

Hi, I wanted to run the packet interceptor using multiprocessing, so I wrote the following codes. When I run the
it it calls the dpi_start function however it doesn't capture any packet.
"""
Interceptor example using ICMP

ADD this rule:
sudo iptables -I INPUT 1 -p icmp -j NFQUEUE --queue-balance 0:2
"""
import time
from multiprocessing import Process
import os
from pypacker import interceptor
from pypacker.layer3 import ip

class TrafficMonitor(object):
    # ICMP Echo request intercepting
    def verdict_cb(self, ll_data, ll_proto_id, data, ctx):
        # ip1 = ip.IP(data)
        ip1 = ip.IP(data)


        print('\n---------->Packet recieved:\n\t source ip:%s , the dst ip:%s, \n\npacket_info:%s '
            %(ip1.src_s, ip1.dst_s, ip1))
        return ip1.bin(), interceptor.NF_ACCEPT

    def dpi_start(self, queue_id):
        print('queue', queue_id)
        ictor = interceptor.Interceptor()
        # queue = [0, 1, 2]
        ictor.start(self.verdict_cb, queue_ids=queue_id)


def start_int():
    monitor = TrafficMonitor()
    proc = Process(target=monitor.dpi_start, args=([0, 1, 2],))
    proc.start()


if __name__ == '__main__':
    while True :
        a = input('press y to start: ')
        if a == 'y':
            start_int()

        elif a == 'x':
            os._exit(0)

I tried this also proc = Process(target=ictor.start, args=(self.verdict_cb, [0, 1, 2],)) the same issue persists, it seems the ictor.start is called but no packet is captured.
However, when I using threading library t = Thread(target=ictor.start, args=(self.verdict_cb, [0, 1, 2],))it works as expected.

I would like to know if packet interceptor has any issue with multiprocessing or I am doing something wrong.
Thank you

I'm working on a project where I would like to modify the TCP payload sort of like a man in the middle. Do you happen to have an example of that? I tried using the interceptor example but I keep getting
ERROR (init): could not dissect or unpack in IP: TypeError("'NoneType' object is not subscriptable",)
I'm assuming that is because I'm somehow not accounting for layer4 stuff since the interceptor example only works at layer3. Any advice/ideas your could give me would be greatly appreciated.

I'm trying to build ip6/tcp packet as following:

from pypacker.layer12 import ethernet
from pypacker.layer3 import ip6
from pypacker.layer4 import tcp

packet = ethernet.Ethernet() + ip6.IP6() + tcp.TCP()

packet.ip6
Output:
IP6(v_fc_flow=60000000, dlen=0, nxt=0, hlim=0, src='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', dst='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', opts=[], handler=tcp)
TCP(sport=DEAD, dport=0, seq=3735928559L, ack=0, off_x2=50, flags=2, win=FFFF, sum=0, urp=0, opts=[], bytes='')

packet.ip6.bin()
Output: '`\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xde\xad\x00\x00\xde\xad\xbe\xef\x00\x00\x00\x00P\x02\xff\xff\x00\x00\x00\x00'

It seems that the IPv6 header is empty, except for the header type (6, at the first byte). No payload length, no next header etc.

Is it a bug or am I missing something?

Thanks,
Yoram

Iteration over every layer starting with lowest layer doesn't work. All time first layer is skipped because _get_bodyhandler() method returns next layer for lowest layer as well.

pkt = ethernet.Ethernet()+ip.IP()+tcp.TCP()
for p in pkt.lowest_layer:
... print(p)
...
IP(v_hl=45, tos=0, len=28, id=0, off=0, ttl=40, p=6, sum=7AD1, src=b'\x00\x00\x00\x00', dst=b'\x00\x00\x00\x00', opts=[], handler=tcp)
TCP(sport=DEAD, dport=0, seq=DEADBEEF, ack=0, off_x2=50, flags=2, win=FFFF, sum=3398, urp=0, opts=[], bytes=b'')

I created pull request with proposal to start iterate from first layer all time. It makes iteration more flexible.
To skip first layer use upper layer or body handler. See examples below

1. Iterate over every layer

pkt = ethernet.Ethernet()+ip.IP()+tcp.TCP()
for p in pkt:
... print(p)
...
Ethernet(dst=b'\xff\xff\xff\xff\xff\xff', src=b'\xff\xff\xff\xff\xff\xff', vlan=[], type=800, handler=ip)
IP(v_hl=45, tos=0, len=28, id=0, off=0, ttl=40, p=6, sum=7AD1, src=b'\x00\x00\x00\x00', dst=b'\x00\x00\x00\x00', opts=[], handler=tcp)
TCP(sport=DEAD, dport=0, seq=DEADBEEF, ack=0, off_x2=50, flags=2, win=FFFF, sum=3398, urp=0, opts=[], bytes=b'')

2. Skip first layer

for p in pkt.upper_layer:
... print(p)
...
IP(v_hl=45, tos=0, len=28, id=0, off=0, ttl=40, p=6, sum=7AD1, src=b'\x00\x00\x00\x00', dst=b'\x00\x00\x00\x00', opts=[], handler=tcp)
TCP(sport=DEAD, dport=0, seq=DEADBEEF, ack=0, off_x2=50, flags=2, win=FFFF, sum=3398, urp=0, opts=[], bytes=b'')

3. Start from highest

for p in pkt.highest_layer:
... print(p)
...
TCP(sport=DEAD, dport=0, seq=DEADBEEF, ack=0, off_x2=50, flags=2, win=FFFF, sum=3398, urp=0, opts=[], bytes=b'')

Thanks,
Nazar

Sender ip and target ip fields length should be defined as 4 bytes instead of 6 bytes

class ARP(pypacker.Packet):
hdr = (
("hrd", "H", ARP_HRD_ETH),
("pro", "H", ARP_PRO_IP),
("hln", "B", 6), # hardware address length
("pln", "B", 4), # protocol address length
("op", "H", ARP_OP_REQUEST),
("sha", "6s", b"\x00" * 6), # sender mac
("spa", "4s", b"\x00" * 4), # sender ip 4 bytes instead of 6
("tha", "6s", b"\x00" * 6), # target mac
("tpa", "4s", b"\x00" * 4) # target ip4 bytes instead of 6

In a custom layer5 protocol, I added a header "length" in the hdr tuple of tuples. It is fixed size (2 octets) and I used the FIELD_FLAG_AUTOUPDATE option. I wrote the code for _update_fields (for current HEAD of master) and for bin() (version 4.0) to calculate the length of the upper_layer body and store that in self.length. For both, the correct length is written to the header of my custom protocol.

The problem is that the TCP.sum is never recalculated to account for the updated length field. I traced the calls, and I see that TCP._update_fields() is only called once and it is before the call to my custom protocol's _update_fields() (or bin() in the case of version 4.0).

Can you recommend an example where a layer567 protocol is layered above TCP, and it has automatically updated simple header fields? I am not doing anything complicated that requires dynamic fields, deactivated fields, or trigger lists.

It should be self.type

I'm trying to run an example "Send and receive packets (send/receive raw bytes)" and I got the following error:

Traceback (most recent call last):
  File "capture.py", line 6, in <module>
    psock = psocket.SocketHndl(mode=psocket.SocketHndl.MODE_LAYER_2, timeout=10)
  File "/home/user/miniconda3/lib/python3.6/site-packages/pypacker/psocket.py", line 46, in __init__
    socket.htons(SocketHndl.ETH_P_ALL))
  File "/home/user/miniconda3/lib/python3.6/socket.py", line 144, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 97] Address family not supported by protocol

I'm using the latest version: pypacker==4.3

System info:

lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.2 LTS
Release:        16.04
Codename:       xenial

Thanks for the library. There aren't many like it for Python 3 yet.

I'm posting this regarding Packet object initialization in pypacker.py. In there, any exceptions are caught and logged internally.

            except Exception as e:
                # TODO: remove to continue parsing
                # raise Exception("%r" % e)
                logger.exception("could not dissect or unpack: %r" % e)

(Around line 409 at time of writing)

Would it be possible to have it raise the exception and not pass over it? I'd like to handle them personally if something goes wrong. Maybe to avoid backward compatibility issues, log-and-pass could be the default, but setting a flag could force it to raise the exception?

I configured py-packer on two computers. I then transfer a large amount of data between the two computers (100MB). the Py-packer on the sending computer stops working at random points (4M - 80M). If I stop/start the py-packer code on the sending side it starts working again. If I stop/start it on the recieving side nothing happens.
Below is the code used on both sides, its pretty much the example from this site. Its very repeatable but I'm not even sure how to start debugging it since nothing crashes. Any advice or suggestions for a starting point? I did try doubling the number of queues and I tried having two differnt sets of queues for incoming and out going data. I made sure to configure the iptables rules to split incoming and outgoing to the different queues as well.
Snippet from when I did the seperate in and out queues

ictor.start(verdict_cb, queue_ids=[0, 1, 2, 3])
ictor2.start(verdict_cb, queue_ids=[4, 5, 6, 7])

Here is the main code

import time
from pypacker import interceptor
from pypacker.layer3 import ip, icmp
from pypacker.layer4 import tcp

# Note I'm not testing if its a TCP packet lbecause the iptables are only routing TCP packets to the queues
def verdict_cb(ll_data, ll_proto_id, data, ctx):
        ip1 = ip.IP(data)
        pktLen = ip1.len
        dataLen = len(ip1.tcp.body_bytes)
        seq = ip1.tcp.seq
        return ip1.bin(), interceptor.NF_ACCEPT

ictor = interceptor.Interceptor()
ictor.start(verdict_cb, queue_ids=[0, 1, 2])
while 1:
    time.sleep(999)

ictor.stop()

While crafting IPv4 packet, following exception was raise.
Code:

        pkt = ip.IP(src_s='127.0.0.1', dst_s='127.0.0.2', tos=0, id=12, off=1024, ttl=64)
        pkt.flags = ip.IP_DF|ip.IP_MF
        pkt.body_bytes = b'12345678'
        raw = pkt.bin()

TraceBack:

Traceback (most recent call last):
  File "/home/user/code/py3/project/unittests/test_ip.py", line 57, in test_match_unicast_true
    raw = pkt.bin()
  File "/home/user/.local/lib/python3.6/site-packages/pypacker/pypacker.py", line 820, in bin
    layer._update_fields()
  File "/home/user/.local/lib/python3.6/site-packages/pypacker/layer3/ip.py", line 229, in _update_fields
    self.sum = in_cksum(self._pack_header())
  File "/home/user/.local/lib/python3.6/site-packages/pypacker/checksum.py", line 49, in in_cksum
    return in_cksum_done(in_cksum_add(0, buf))
  File "/home/user/.local/lib/python3.6/site-packages/pypacker/checksum.py", line 23, in in_cksum_add
    n = len(buf)
TypeError: object of type 'NoneType' has no len()

Something I did wrong, or its some corner case. Hoe to access error object from pypacker.PyPacket

Not sure if anyone else is using the NTP one much, but I was trying to get delay and dispersion and realized they appear to be declared incorrectly. With the format they are declared here I haven't figured out how to break them up to get the info correctly.

Current:
class NTP(pypacker.Packet):
hdr = (
("flags", "B", 0x1c), # li | v | mode
("stratum", "B", 0x2),
("interval", "B", 0x4),
("precision", "B", 0xe9),
("delay", "I", 0),
("dispersion", "I", 0),
("id", "4s", b"\x00\x01\x02\x03"),
# timestamps: [seconds since 1.1.1900 | fraction of seconds]
("update_time", "8s", b"\x00" * 8),
("originate_time", "8s", b"" * 8),
("receive_time", "8s", b"" * 8),
("transmit_time", "8s", b"" * 8)
)

delay and dispersion aren't "I", they're each NTP Short which is just one more odd thing in NTP:
The NTP Short Format is a uint32_t with the first 16 bits being the seconds (i.e seconds = format >> 16) and the other 16 bits being the fraction of a second (i.e fraction = format & 0xFFFF)

All the *_time are NTP Long, where the first 32 are Seconds and 2nd 32 are Fraction
https://www.meinbergglobal.com/english/info/ntp-packet.htm

Could potentially be updated to something like this:
class NTP(pypacker.Packet):
hdr = (
("flags", "B", 0x1c), # li (2) | v (3) | mode (3)
("stratum", "B", 0x2),
("interval", "B", 0x4),
("precision", "B", 0xe9),
("delay","4s", b"\x00" *4),
("dispersion", "4s", b"\x00" *4),
("id", "4s", b"\x00\x01\x02\x03"),
# timestamps: [seconds since 1.1.1900 | fraction of seconds]
("update_time_sec", "4s", b"\x00" * 4),
("update_time_frac", "4s", b"\x00" * 4),
("originate_time_sec", "4s", b"\x00" * 4),
("originate_time_frac", "4s", b"\x00" * 4),
("receive_time_sec", "4s", b"\x00" * 4),
("receive_time_frac", "4s", b"\x00" * 4),
("transmit_time_sec", "4s", b"\x00" * 4),
("transmit_time_frac", "4s", b"\x00" * 4),
)

Hack on the time since it is from 1.1.1900, but doing the following in my code to covert to utc format

def ntpTimeConvert(date):
value = ''
offset = 2208988800
time = int.from_bytes(date[0:4], "big")
if time > offset:
time = time - offset

value = datetime.utcfromtimestamp(time.strftime('%Y-%m%dT%H:%M:%S')

return value

Using this package for the first time earlier today: for the code below, the ICMP Echo len returns a value great than the ICMP len and throws an exception when trying to print the Echo. Is this unexpected or am I misunderstanding the API?

from pypacker import psocket
from pypacker.layer3 import ip

psock = psocket.SocketHndl(iface_name='lo', timeout=10)

p = psock.recvp()[0]
print(len(p.ip.icmp), ":", p.ip.icmp)
print(len(p.ip.icmp.echo), ":", p.ip.icmp.echo)

psock.close()

Testing the above using:

$ ping -4 -c 1 -s 0 127.0.0.1

Causes:

$ sudo python test2.py
8 : ICMP(type=8, code=0, sum=BAFB, handler=echo)
12 : Traceback (most recent call last):
  File "/home/sam/.virtualenvs/foo/lib/python3.6/site-packages/pypacker/pypacker.py", line 581, in _unpack
    header_unpacked = self._header_format.unpack(self._header_cached)
struct.error: unpack requires a bytes object of length 12

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test2.py", line 8, in <module>
    print(len(p.ip.icmp.echo), ":", p.ip.icmp.echo)
  File "/home/sam/.virtualenvs/foo/lib/python3.6/site-packages/pypacker/pypacker.py", line 546, in __str__
    return self._summarize()
  File "/home/sam/.virtualenvs/foo/lib/python3.6/site-packages/pypacker/pypacker.py", line 516, in _summarize
    self._unpack()
  File "/home/sam/.virtualenvs/foo/lib/python3.6/site-packages/pypacker/pypacker.py", line 586, in _unpack
    self._header_field_names, self._header_cached, len(self._header_cached)))
Exception: could not unpack in: Echo, format: b'>HHd', names: ['_id', '_seq', '_ts'], value to unpack: b'=\x03\x00\x01' (4 bytes), not enough bytes?

$ python --version
Python 3.6.1
$ pip show pypacker | grep Version                       
Version: 4.0

In a custom layer5 protocol, I added variable padding to ensure that the number of bytes of the payload (body bytes of the TCP segment) is a multiple of 4. I followed an example and did this by adding the pad byte string to the end in the protocol's bin() implementation. The IP.len is not recomputed after this, so it is always off by the size of the variable pad.

I tried both released version 4.0 and the current HEAD of master.

When try to enable promiscuous mode in Linux via SocketHndl class AttributeError exception is raised.

from pypacker import psocket
sock_l2 = psocket.SocketHndl(iface_name="eth0", promisc=True)
Traceback (most recent call last):
File "", line 1, in
File "/src/pypacker/pypacker/psocket.py", line 48, in init
self._socket_recv.setsockopt(socket.SIO_RCVALL, socket.RCVALL_ON)
AttributeError: module 'socket' has no attribute 'SIO_RCVALL'

This is an informational ticket to inform about the relocation of the pypacker github repository (If you didn't already see it in the README.md):

Please note: This respository has become staled due to relocation to GitLab. Visit https://gitlab.com/mike01/pypacker for up-to-date versions.

I'm trying to get the data from ICMP protocol. From the documentation, accessing the body of a header (such as an IP header containing a UDP packet in the body) you would write

eth[ip.IP][udp.UDP].sport

but with an ICMP message, the UDP packet is in the body of the IP header in the ICMP and trying something like

eth[icmp.ICMP][ip.IP][udp.UDP].sport

does not work because it is a NoneType

Any solution?

on running the given example code the following error is showing up
File "pypacker_exmpls.py", line 1, in
import pypacker.pypacker as pypacker
File "/usr/local/lib/python2.7/dist-packages/pypacker/pypacker.py", line 31
class Packet(object, metaclass=MetaPacket):

The error message reads:
OSError: libnetfilter_queue.so.1: cannot open shared object file: No such file or directory

Replace string type with byte type for default values in packet field. In certain modules rfb.py, mb.py

Example:
pypacker/pypacker/layer567/smb.py

class SMB(pypacker.Packet):
hdr = [
("proto", "4s", b""),
("cmd", "B", 0),
("err", "I", 0),
("flags1", "B", 0),
("flags2", "B", 0),
("pad", "6s", ""), # replace with b""
("tid", "H", 0),
("pid", "H", 0),
("uid", "H", 0),
("mid", "H", 0)
]

pypacker/pypacker/layer567/rfb.py
class SetPixelFormat(pypacker.Packet):
hdr = (
("pad", "3s", ""), # replace with b""
("pixel_fmt", "16s", "") # replace with b""
)

When I try to use Pcap Next generation format file , pypacker gives me back this exception:

raise ValueError("invalid tcpdump header, magic value: %s" % self.__fhdr.magic)
ValueError: invalid tcpdump header, magic value: 168627466

[fix] I think, for now Pypacker don't support next gen Pcap file. [fix] , sorry it supports but i dont know why i can't use pcapng files. With Wireshark , there are no errors when i open pcapng file.

When I am trying to read packets from a pcap. file and convert them with ethernet.Ethernet() I am running into the following error:

ERROR (__init__): could not dissect in SSL: error('unpack requires a bytes object of length 2',)
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/pypacker-4.0-py3.5.egg/pypacker/pypacker.py", line 169, in __init__
    header_len = self._dissect(args[0])
  File "/usr/local/lib/python3.5/dist-packages/pypacker-4.0-py3.5.egg/pypacker/layer4/ssl.py", line 146, in _dissect
    record_len = unpack_H(buf[offset + 3: offset + 5])[0]
struct.error: unpack requires a bytes object of length 2

And the same code in ipython3 gives the following:

ValueError: not enough values to unpack (expected 2, got 1)

The code is as follows:

from pypacker import ppcap
from pypacker.layer12 import ethernet
from pypacker.layer3 import ip
from pypacker.layer4 import tcp,udp
for ts, raw in ppcap.Reader(filename=pcapfile):
    eth = ethernet.Ethernet(raw)   
        #create the keys for IP UDP/TCP flows
        if eth[ip.IP] is not None:
            print(eth)

The pcap file is available via the link:
https://yadi.sk/d/p-ylrJSp3ToLAd

update:
the binary form of a packet the error is being thrown from:

b'\xf4\xec8\xa8\xa0\xf2\x1coeN7\r\x08\x00E\x00\x05\x8cQk@\x00\x80\x06\xd7&\xc0\xa8\x01d\n\x00\x00\xce\xc5\xb9\x00PU\x06\xeaF\xf6g\xe5DP\x18\x01\x00\x03j\x00\x006\x007\x004\x00C\x001\x008\x00C\x007\x005\x009\x005\x00A\x00D\x003\x006\x00C\x00B\x005\x004\x00<\x00/\x00P\x00r\x00o\x00p\x00e\x00r\x00t\x00y\x00>\x00<\x00/\x00H\x00o\x00o\x00k\x002\x00>\x00<\x00/\x00H\x00o\x00o\x00k\x00s\x00>\x00<\x00P\x00a\x00y\x00l\x00o\x00a\x00d\x00 \x00T\x00y\x00p\x00e\x00=\x00"\x00i\x00n\x00l\x00i\x00n\x00e\x00"\x00/\x00>\x00<\x00T\x00a\x00r\x00g\x00e\x00t\x00H\x00o\x00s\x00t\x00>\x00S\x00R\x00V\x00-\x00S\x00C\x00C\x00M\x00.\x00d\x00c\x00k\x00a\x00i\x00.\x00r\x00u\x00<\x00/\x00T\x00a\x00r\x00g\x00e\x00t\x00H\x00o\x00s\x00t\x00>\x00<\x00T\x00a\x00r\x00g\x00e\x00t\x00E\x00n\x00d\x00p\x00o\x00i\x00n\x00t\x00>\x00M\x00P\x00_\x00R\x00e\x00l\x00a\x00y\x00E\x00n\x00d\x00p\x00o\x00i\x00n\x00t\x00<\x00/\x00T\x00a\x00r\x00g\x00e\x00t\x00E\x00n\x00d\x00p\x00o\x00i\x00n\x00t\x00>\x00<\x00R\x00e\x00p\x00l\x00y\x00M\x00o\x00d\x00e\x00>\x00A\x00s\x00y\x00n\x00c\x00<\x00/\x00R\x00e\x00p\x00l\x00y\x00M\x00o\x00d\x00e\x00>\x00<\x00P\x00r\x00o\x00t\x00o\x00c\x00o\x00l\x00>\x00h\x00t\x00t\x00p\x00<\x00/\x00P\x00r\x00o\x00t\x00o\x00c\x00o\x00l\x00>\x00<\x00/\x00M\x00s\x00g\x00>\x00\r\n--aAbBcCdDv1234567890VxXyYzZ\r\ncontent-type: application/octet-stream\r\n\r\nx\x9c\xed\x96[O\x1aQ\x10\xc7\xe7\xa3\x18^|h\x80\xdde+\xb4A\x1b\xb9l%\xd5\xd6Hm_\xfaB\x04\xd1\x14\xc4\x08^\xe8\x87o\xfb\x9b9\x0b\xec\xc5\x8a\xe9\xed\x89\x90\xb3;g.\xff3\xb73K]Nd \xd72\x91\x1b\x99\xc9\x9e\xd4S\xfb\x03\xe8\x9e\xf4y\xde\x98\xacc\xf4\x15\x92K9g\x9d!Uz\x02O\xe5G\xec\xcf\xe4\x02\xce\x15z\xcai\xca\x88\x9d\xb3\xe9\xf0\x9c\xf2\xee\xc1\x1b\xc1\xeb\xa3\xe1\xa3S^\xab\x95\xc6\xf9(s<\x1c<j\xbd\x92e\xcfn\xc1{+\xa7F\xbd\x96\x90g\x1b\x8d\x80g[<)JE\x1aP>T(\x91\xd4x\x16\xe1\xa8\x9eoT\x15\xed\x1a;\x8f_SvX\x11+\xef}+w\xf6\'\xcb\xdft\x99\xa7\x97R2\x94\x12x\x01\xe7\xbe\x82\xf2\x8d\xe3\xe5\xf0\xb2\xb6uy\x0fg\x86?\x1d\xf9 ]v=\x19[\xbc\'\xc4\xde\xc5S\x0f\xfc"\xcb\x03Yw.G\xbf\xb2RO\'V\xd5cxC\xe3\xd5\x88n\'\xf6$+\xabc=\xb7\xfa\x0c@h\xf1<Gv\x8b\xc739D\xdf\xc5\xaf\xd1\x84\xc4\xa5\x18\xcf\xd1W\xbd|\xef\x94\xd7\xf6[\xb2W[\x16a\x0f\xe9\x88\xf3\xb2\xd2\xa6\xd9\xccb\xb4=\xbc\x9a\x808\x93{,n\xe0~\x91\x07\xcb\x7f\xc0:2$\xcd\xbb\xfa2L\xc9Z\xe6A\xcf\xbc{\n?}\xfa\xaa+#\x8b}\x94\xb3O\xf6\xad;C\xe9\xc0\xeaW\xb3Z\x06t\xa0\xcf/\xe4]\xa1>%\xa3+q\xdf\xbc\x88\xf5\x14we_\xcf\xf5\x8fo\xbd\xa7Z\xf9\xce\x8a\xcc\x97\xb1Y\'5\xd3\xfc\xac\xe7\xf9\xbc\x97\x9f\x9c!\xe5\xcc\xc4\x11\xf9\xf1\xbd.o\xc8\xf1\x18\x84-\xb9K\xf9\xb5+\xdb\xb1\'\xdb\xc84\xbbg\xd6\x93\xae2*=%w\x11\x9d\xee\x93\x93mp\xb2\xb9\xdf\xcc\xb4\xcdL\xdb\xcc\xb4\xcdL\xfb\x973-)kX\x7f\xcd\x8d\x7f\x0cGw\xb7T\xd9M\x81s\xf6[\x96\xd91\x16=8\xf3e\xe7\xeeJ\x81\x93;\xe4\xadHf\x02z\xad\x80n\x1a#\xa9\xfb9u\xab\xf2\xba\xe9l\xa8\x85\x8b\x7f\xb1\xf2\x16\x87\xe6\xd1\x90\x9d\xbb\x1b\xce\xc6\xf5}!\xbe)\xe9\x1e\x8b,G\x83Dl\x0b\x8e\xe6p\n\xaav\xdc\xf5\xb2\xc3\x15\xf1\x10jh}XDS\xe9\t\'\xce\xb0\x1d\x9b\xce%\xb4\xe6\xb2\x90@{N,\x0b\xddd\x8e\xee\xf1`\x95\xa3\x12\xef\x07VR\xbb\xcb\xfb[\xac\xad]\xa8\xbd\x18\x18\xa2\xc6{\x80t\x1a\xfb\xe3\xee\xea(q\x97\x167g+\xc1\xd3\xe9\xbe\xe0\xeb$p\xb8:\x7f\xab\xfc\xda\x96O\x9d\xbd\x81\xf1t27\x98l\xb5\x98\xd76\x9ez\xa13/\x92}\x10\xaa\xbc}*\xd00\xbfO\xf1G\xcfIwMS\xde\xa1\xdb\xe1^\x9f\xa0\xdd\xc0\xd7\xafHoy^\xc4z}b\xb83\x84\xae\xf9\xe6n\xe8\xe5\x12\xe3\xb1{\xe9\xd97\xa3\x1a\xcfv/s/\x15\xabM^\xfbkqV\xf7;\xb09\x9c\xc7q>\xcd\xf0xj(\x958\xff\xe5gV\xe0\xef\xd5j\x9f\xfc\x85\xf1W\xb4j\xb5\xf2-\x8a\x96\xd5\xaa\x82\xc7\x91\xd5J\xe7f`_\xa5\x00N\xdb\xbe\xbd!R\xfd5\xcd\xff\xffU\xab\x1d\xeb\x0e\xcffF\xf8\x07\xb5r8\x15Cy\n\'[\xab\xe07jU^;I\x9c\xd6\xe3\x134?\x91\x93S7\xfb\x0f\xf3\'\x1e\xa4j\x16\r\n--aAbBcCdDv1234567890VxXyYzZ--'

Although dpkt seem to process it correctly.

I am writing some pcap reader

from pypacker import ppcap

for ts, buf in preader:
    cache.manager(buf, ts)

Unfortunatelly for some pcaps I am getting

Traceback (most recent call last):
  File "C:/Users/p001/PycharmProjects/pyprobe/pcapreader.py", line 20, in <module>
    preader = ppcap.Reader(filename="ez.pcap")
  File "C:\Python\Python3.7.3\lib\site-packages\pypacker\ppcap.py", line 305, in __init__
    super().__init__(filename, PcapHandler.MODE_READ, **initdata)
  File "C:\Python\Python3.7.3\lib\site-packages\pypacker\ppcap.py", line 239, in __init__
    raise Exception("no matching handler found")
Exception: no matching handler found

Thanks,
Pawe

When I try install pypacker via pip it will end with error message:
Could not find a version that satisfies the requirement pypacker (from versions: ) No matching distribution found for pypacker

I tried it on Linux Manjaro with Python 3.6.0 and on Debian with Python 3.4.2 both end up with the same results.

Thanks for this library, it has certainly made my life easier! but I have this little problem..

I captured an IP Packet on an android device and tried to parse TCP and I got a invalid header length error.
here is the packet's byte string:

b'E\x00\x00<F\x84@\x00@\x06\xa0)\n\x00\x02\x00.\x04\x1a\x0b\xcd\x9a\x00P\ri\x83\x16\x00\x00\x00\x00\xa0\x02|8Sa\x00\x00\x02\x04\x02\x12\x04\x02\x08\n\x01<\xc8T\x00\x00\x00\x00\x01\x03\x03\x06''

and here is the code I use to parse it

from pypacker.layer3 import ip
from pypacker.layer4 import tcp

pbytes = #bytestring above
ipp = ip.IP(pbytes)
if ipp[tcp.TCP] is not None:
        print(tcp.TCP(pbytes))

Whenever, It gets to the "print(tcp.TCP(body))" line, it gives this error:

ERROR (__init__): could not dissect, trying to unpack after all..
Traceback (most recent call last):
  File "C:\Python33\lib\site-packages\pypacker\pypacker.py", line 257, in __init__
    self._dissect(args[0])
  File "C:\Python33\lib\site-packages\pypacker\layer4\tcp.py", line 142, in _dissect
    raise pypacker.UnpackError("invalid header length")

Please, can you help me fix this?

after use ethpkt=ethernet.Ethernet(something) to create a ethernet pkt . Why must use tcppkt=ethpkt[tcp.TCP] to get tcppackt. When something is a pcap. it should know it is tcp or udp . or it is ipv4 or ipv6 , How can we just use to ethpkt[ethpkt.l3proto] to get it ipv4 or ipv4 , or ethpkt[ethpkt.l4poroto] to get tcp or udp ? Otherwise , i should try ethpkt[udp.UDP] to get None then try tcp or is there a good methond detect the l3 l4 layer proto in ethpkt?

Hi,

Unrelated note: First of all, I am sorry that I didn't put this issue in gitlab. However, Gitlab currently seems to have quite strong restrictions with allowed domain names for email. Apparently my email provider(s) aren't on this whitelist, lol.

When I create a psocket with Pypacker the following exception is thrown:

AttributeError: module 'socket' has no attribute 'AF_PACKET'
Apparently, there is no such thing as socket.AF_PACKET for windows and the pypacker.psocket seems to rely on this.

Pypacker works fine with Linux, however for my use case I am restricted to Windows. In the FAQ it is stated that it should run on Windows as well. Therefore I think this should be an issue.

I am trying to write an UDP Sniffer / UDP Forwarding Script with Pypacker. Is there a workaround for the usage of pypacker.psocket or socket.AF_PACKET?

This would be a stripped down example of what I am currently doing:

from pypacker import psocket
from pypacker.layer12 import ethernet
from pypacker.layer3 import ip
from pypacker.layer4 import udp

def sniff(interface_name, port):
    psock = psocket.SocketHndl(iface_name=interface_name)
    for raw_bytes in psock:
        pypkt = ethernet.Ethernet(raw_bytes)
        if pypkt[ethernet.Ethernet, ip.IP, udp.UDP] is not None:
            if pypkt[udp.UDP].sport == port:
                print("packet: %r" % pypkt)

Thanks!

Got another bug for you. This one is coming from the BGP module. It looks like a function isn't being called properly. Here's the error message:

ERROR (__init__): could not dissect or unpack: TypeError('_unpack() takes 1 positional argument but 2 were given',)
Traceback (most recent call last):
  File "/home/user/python3/lib/python3.4/site-packages/pypacker/pypacker.py", line 401, in __init__
    header_len = self._dissect(args[0])
  File "/home/user/python3/lib/python3.4/site-packages/pypacker/layer567/bgp.py", line 175, in _dissect
    pypacker.Packet._unpack(self, buf[:4])
TypeError: _unpack() takes 1 positional argument but 2 were given

I was running the script on the bgp.pcap file available here.

Here is the one-off script I'm working with. Again, it's not very pretty, but I'm not really going anywhere specific with it.

import pcapy
import os
from pypacker.layer12 import ethernet
from pypacker.layer4 import tcp
from pypacker.layer567 import bgp

#https://wiki.wireshark.org/SampleCaptures#Routing_Protocols
PCAPFILE = os.path.join(os.environ['HOME'], "pcap", "bgp.pcap")
BPF = "tcp and port 179"

capture = pcapy.open_offline(PCAPFILE)
capture.setfilter(BPF)

while True:
    try:
        header, packet = capture.next()
    except pcapy.PcapError:
        break

    pkt = ethernet.Ethernet(packet)
    bgp_pkt = pkt.highest_layer

    if isinstance(bgp_pkt, tcp.TCP):
        continue

    if isinstance(bgp_pkt, bgp.BGP):
        print("BGP")

    elif isinstance(bgp_pkt, bgp.BGP.Open):
        print("OPEN")

    elif isinstance(bgp_pkt, bgp.BGP.Update):
        print("UPDATE")

    else:
        print("OTHER", type(bgp_pkt))

And here is the full output:

> python3 bgp_error_test.py 
OPEN
OPEN
BGP
BGP
OTHER <class 'pypacker.layer567.bgp.BGP.Keepalive'>
ERROR (__init__): could not dissect or unpack: TypeError('_unpack() takes 1 positional argument but 2 were given',)
Traceback (most recent call last):
  File "/home/joshuae/python3/lib/python3.4/site-packages/pypacker/pypacker.py", line 401, in __init__
    header_len = self._dissect(args[0])
  File "/home/joshuae/python3/lib/python3.4/site-packages/pypacker/layer567/bgp.py", line 175, in _dissect
    pypacker.Packet._unpack(self, buf[:4])
TypeError: _unpack() takes 1 positional argument but 2 were given
UPDATE
BGP