ocaml-community / biniou Goto Github PK

View Code? Open in Web Editor NEW

43.0 43.0 17.0 181 KB

Extensible binary data format, like JSON but faster

Home Page: https://mjambon.github.io/atdgen-doc

License: Other

Makefile 0.25% OCaml 99.75%

biniou's People

Contributors

Stargazers

Watchers

Forkers

pmundkur hhugo dweil unixjunkie kazbanov bsansouci rgrinberg kit-ty-kate alan-j-hu roddyyaga andrianmarinov lexifi idkjs eduardorfs patricoferris isabella232 feret

biniou's Issues

Comparison to Similar Tech

Reading the README it is not clear how this compares with protobufs or avro or any of the other similar technologies out there. A comparison would be really nice :)

Compilation failed on Debian

Hello,

I fail to install biniou using Opam on Debian Stretch 64bits. I have no clue what the error messages mean so I hope you can help me.

The following actions will be performed:
  - install biniou 1.0.13

=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[biniou] Archive in cache

=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[biniou: make] Command started
[ERROR] The compilation of biniou failed at "make".
[biniou: ocamlfind remove] Command started

#=== ERROR while installing biniou.1.0.13 =====================================#
# opam-version         1.2.2
# os                   linux
# command              make
# path                 /home/cauderlier/.opam/4.03.0/build/biniou.1.0.13
# compiler             4.03.0
# exit-code            2
# env-file             /home/cauderlier/.opam/4.03.0/build/biniou.1.0.13/biniou-18579-1872ec.env
# stdout-file          /home/cauderlier/.opam/4.03.0/build/biniou.1.0.13/biniou-18579-1872ec.out
# stderr-file          /home/cauderlier/.opam/4.03.0/build/biniou.1.0.13/biniou-18579-1872ec.err
### stdout ###
# [...]
# ocamlfind ocamlc -a -g -annot -bin-annot -o biniou.cma \
#       -package "easy-format" bi_util.mli bi_util.ml bi_share.mli bi_share.ml bi_outbuf.mli bi_outbuf.ml bi_inbuf.mli bi_inbuf.ml bi_vint.mli bi_vint.ml bi_io.mli bi_io.ml bi_dump.ml bi_stream.mli bi_stream.ml
# ocamlfind ocamlc -compat-32 -o bdump.byte -g -annot -bin-annot \
#       -package easy-format -linkpkg biniou.cma bdump.ml
# ocamlfind ocamlopt -a -g -annot -bin-annot \
#       -o biniou.cmxa -package "easy-format" bi_util.mli bi_util.ml bi_share.mli bi_share.ml bi_outbuf.mli bi_outbuf.ml bi_inbuf.mli bi_inbuf.ml bi_vint.mli bi_vint.ml bi_io.mli bi_io.ml bi_dump.ml bi_stream.mli bi_stream.ml
# ocamlfind ocamlopt -shared -linkall -I . -o biniou.cmxs biniou.cmxa
# ocamlfind ocamlopt -o bdump.native -g -annot -bin-annot \
#       -package easy-format -linkpkg biniou.cmxa bdump.ml
# Makefile:91: recipe for target 'bdump.native' failed
### stderr ###
# [...]
# /usr/bin/ld: /home/cauderlier/.opam/4.03.0/lib/ocaml/libasmrun.a(fail.o): relocation R_X86_64_32 against symbol `caml_exn_Failure' can not be used when making a shared object; recompile with -fPIC
# /usr/bin/ld: /home/cauderlier/.opam/4.03.0/lib/ocaml/libasmrun.a(freelist.o): relocation R_X86_64_32S against `.bss' can not be used when making a shared object; recompile with -fPIC
# /usr/bin/ld: /home/cauderlier/.opam/4.03.0/lib/ocaml/libasmrun.a(misc.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
# /usr/bin/ld: /home/cauderlier/.opam/4.03.0/lib/ocaml/libasmrun.a(parsing.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
# /usr/bin/ld: /home/cauderlier/.opam/4.03.0/lib/ocaml/libasmrun.a(printexc.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
# /usr/bin/ld: final link failed: Nonrepresentable section on output
# collect2: error: ld returned 1 exit status
# File "caml_startup", line 1:
# Error: Error during linking
# make: *** [bdump.native] Error 2



=-=- Error report -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The following actions failed
  - install biniou 1.0.13
No changes have been performed

A float comparison fails on ppc64 on line 264 of the test_biniou file. As it can be seen from the log below, I added a print to the respective line in the test and it fails although both numbers seem equal.

$ ./test_biniou
x = 1.001470 y = 1.001470
Fatal error: exception File "bi_io.ml", line 265, characters 4-10: Assertion failed

Add output to OO channel

See http://www.ocaml-programming.de/rec/IO-Classes.html
Patch: http://paste.in.ua/4026/

Portability issue with float serialization

Several people have tried to use Yojson, which depends on Biniou, with Js_of_ocaml. However, Biniou uses an unportable hack to serialize floats. In particular, the expression "String.unsafe_get (Obj.magic 1.0) 0" is unconditionnally executed and fails when compiled to Javascript. Maybe the function Int64.of_float could be used instead to access the binary representation of floats?

Determination of $(PREFIX) is wrong

$(which ocamlfind) gives the location of ocamlfind; when using the OCAMLFIND_TOOLCHAIN environment variable, this is not the same as installation location. Instead, $(shell ocamlfind ocamlc -where)/../../bin can be used (or again, simply include Makefile.config).

no float32 fixed-length type in the specification

Why?

BEST=$(shell ...) is broken on OS X

I'm not sure why exactly, but it's empty. Please consider using OCaml's Makefile.config instead of doing this fragile shell magic.

The workaround is to use make BEST=.native.

Compilation broken on Windows

This is what I get when doing opam install biniou with the official installer:

#=== ERROR while installing biniou.1.0.12 =====================================#
# opam-version 1.2.2
# os           cygwin
# command      install -m 0755 /home/mrm/.opam/system/build/biniou.1.0.12/bdump.byte /home/mrm/.opam/system/bin/bdump
# path         /home/mrm/.opam/system/build/biniou.1.0.12
# exit-code    1
# env-file     /home/mrm/.opam/log/log-4316-4197d2.env
# stdout-file  /home/mrm/.opam/log/log-4316-4197d2.out
# stderr-file  /home/mrm/.opam/log/log-4316-4197d2.err
### stderr ###
# install: cannot create regular file '/home/mrm/.opam/system/bin/bdump': File exists

I had to pin biniou.1.0.9, which has no build issues.

New module to export the untagged_read/write functions

Hello,

I think it may be useful to have a module
so that people can directly read/write
untagged values, without
having to go through biniou's output buffer
(and associated data copy).

Instead of using an output buffer, these
functions would take an in or out_channel.

I'd like to expose in such module all the
read/write_untagged_* functions but
using channels for these ones.

Regards,
F.

reading float32 from C/C++ code

Hello Martin,

I have a float that was was written to disk as an untagged float32
on the OCaml side.

I tried a lot of things but can't manage to decode
the values from some C code.

Do you have some any hint or example?

Thanks,
F.

Failed to install on OS X 10.12.6

Wasn't able to install the latest version or version 1.1.0 with opam. Was able to install 1.0.13.

> opam install biniou
The following actions will be performed:
  ∗  install biniou 1.2.0

=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[biniou] Archive in cache

=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[ERROR] The compilation of biniou failed at "jbuilder build -p biniou -j 4".

#=== ERROR while installing biniou.1.2.0 ======================================#
# opam-version 1.2.2
# os           darwin
# command      jbuilder build -p biniou -j 4
# path         /Users/ben/.opam/system/build/biniou.1.2.0
# compiler     system (4.05.0)
# exit-code    127
# env-file     /Users/ben/.opam/system/build/biniou.1.2.0/biniou-10762-6d07ae.env
# stdout-file  /Users/ben/.opam/system/build/biniou.1.2.0/biniou-10762-6d07ae.out
# stderr-file  /Users/ben/.opam/system/build/biniou.1.2.0/biniou-10762-6d07ae.err



=-=- Error report -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The following actions failed
  ∗  install biniou 1.2.0
No changes have been performed

> opam list
# Installed packages for system:
base-bigarray        base  Bigarray library distributed with the OCaml compiler
base-bytes           base  Bytes library distributed with the OCaml compiler
base-threads         base  Threads library distributed with the OCaml compiler
base-unix            base  Unix library distributed with the OCaml compiler
conf-m4                 1  Virtual package relying on m4
conf-which              1  Virtual package relying on which
cppo                1.5.0  Equivalent of the C preprocessor for OCaml programs
easy-format         1.2.0  High-level and functional interface to the Format module 
jbuilder       1.0+beta12  Fast, portable and opinionated build system
ocamlbuild         0.11.0  OCamlbuild is a build system with builtin rules to easily
ocamlfind           1.7.3  A library manager for OCaml