guyingbo / iofree

iofree is an easy-to-use and powerful library to help you implement network protocols and binary parsers.

iofree

iofree is an easy-to-use and powerful library to help you implement network protocols and binary parsers.

Installation

pip install iofree

Advantages

Using iofree, you can:

• define network protocols and file format in a clear and precise manner
• parse both binary streams and files

Documentation

Basic Usage

>>> from iofree import schema
>>> schema.uint8(1)
b'\x01'
>>> schema.uint32be(3)
b'\x00\x00\x00\x03'
>>> schema.uint32be.parse(b'\x00\x00\x00\x03')
3

Tutorial 1: a simple parser

>>> class Simple(schema.BinarySchema):
...     a = schema.uint8
...     b = schema.uint32be # "be" for big-endian
...
>>> Simple(1, 3).binary
b'\x01\x00\x00\x00\x03'
>>> binary = _
>>> Simple.parse(binary)
<Simple(a=1, b=3)>

Built-in units:

commonly used number units:

• int8 uint8
• int16 int16be uint16 uint16be
• int24 int24be uint24 uint24be
• int32 int32be uint32 uint32be
• int64 int64be uint64 uint64be
• float16 float16be
• float32 float32be
• float64 float64be

simple units:

• Bytes
• String
• EndWith

composite units:

• LengthPrefixedBytes
• LengthPrefixedString
• LengthPrefixedObjectList
• LengthPrefixedObject
• MustEqual
• Switch
• SizedIntEnum
• Convert
• Group

API docs

Here is a real life example definition of socks5 client request, you can see the following code snippet:

class Socks5ClientRequest(schema.BinarySchema):
    ver = schema.MustEqual(schema.uint8, 5)
    cmd = schema.SizedIntEnum(schema.uint8, Cmd)
    rsv = schema.MustEqual(schema.uint8, 0)
    addr = Addr

Tutorial 2: define socks5 address format

In [1]: import socket
   ...: from iofree import schema
   ...:
   ...:
   ...: class Addr(schema.BinarySchema):
   ...:     atyp: int = schema.uint8
   ...:     host: str = schema.Switch(
   ...:         "atyp",
   ...:         {
   ...:             1: schema.Convert(
   ...:                 schema.Bytes(4), encode=socket.inet_aton, decode=socket.inet_ntoa
   ...:
   ...:             ),
   ...:             4: schema.Convert(
   ...:                 schema.Bytes(16),
   ...:                 encode=lambda x: socket.inet_pton(socket.AF_INET6, x),
   ...:                 decode=lambda x: socket.inet_ntop(socket.AF_INET6, x),
   ...:             ),
   ...:             3: schema.LengthPrefixedString(schema.uint8),
   ...:         },
   ...:     )
   ...:     port: int = schema.uint16be
   ...:

In [2]: addr = Addr(1, '172.16.1.20', 80)

In [3]: addr
Out[3]: <Addr(atyp=1, host='172.16.1.20', port=80)>

In [4]: addr.binary
Out[4]: b'\x01\xac\x10\x01\x14\x00P'

In [5]: Addr.parse(addr.binary)
Out[5]: <Addr(atyp=1, host='172.16.1.20', port=80)>

A complete socks5 Addr definition

Projects using iofree

• Shadowproxy
  • socks5 models and socks5 protocol
  • shadowsocks parser
  • shadowsocks aead parser
• python tls1.3
  • TLS1.3 models and protocol

References

iofree parser is inspired by project ohneio.

iofree's Issues

Compare to [Construct](https://github.com/construct/construct)

It is not immediately clear how and if it is better than Construct.

Use iofree to verify the initialization and connection process defined in the rfc1928 for a SOCKS5 Proxy.

Hi,

On Ubuntu 20.04, I've Tor listening on 127.0.0.1:9050 as a socks5 proxy. I want to check the initialization and connection process of it with iofree when I'm using the proxy for access some websites, say, by curl like following:

$ curl -v -x socks5://127.0.0.1:9050 http://clients3.google.com/generate_204

Any hints for this problem?

Regards

Use iofree to check/debug Controller Area Network (CAN bus) protocol.

Hi,

Currently, I'm learning the apollo autonomous driving platform. The CAN protocol is used for communication between the chassis and the IPC. So I want to know whether I can use iofree to check/debug Controller Area Network (CAN bus) protocol for this case.

Regards,
HY

Bitfield Please？

很pythonic的库，不过缺失了bitfield功能，有所遗憾。可否在下一版本实现呢？多谢！