yoriyuki / camomile Goto Github PK
View Code? Open in Web Editor NEWA Unicode library for OCaml
License: Other
A Unicode library for OCaml
License: Other
ocaml-gettext uses camomile. However the latest version of Camomile doesn't work - a Not_found
exception gets thrown deep inside camomile.
An easy reproduce in ocaml-gettext is to simply run:
OCAMLRUNPARAM=b ../_build/bin/ocaml-gettext --action compile --compile-output fr.mo fr.po
Fatal error: exception Not_found
Raised at file "Camomile/internal/database.ml", line 50, characters 52-67
Called from file "Camomile/internal/unidata.ml" (inlined), line 187, characters 2-46
Called from file "Camomile/public/uCharInfo.ml", line 307, characters 2-30
Called from file "camomileLibrary.mlp", line 204, characters 22-44
Called from file "camomileLibrary.mlp", line 168, characters 0-1023
Called from file "Camomile/camomileLibraryDefault.ml", line 37, characters 18-46
Output from opam install camomile
:
#=== ERROR while installing camomile.1.0.0 ====================================#
# opam-version 1.2.2
# os darwin
# command jbuilder build -p camomile -j 4
# path /Users/wczekalski/.opam/4.04.0_ios/build/camomile.1.0.0
# compiler 4.04.0
# exit-code 1
# env-file /Users/wczekalski/.opam/4.04.0_ios/build/camomile.1.0.0/camomile-4894-d2c37b.env
# stdout-file /Users/wczekalski/.opam/4.04.0_ios/build/camomile.1.0.0/camomile-4894-d2c37b.out
# stderr-file /Users/wczekalski/.opam/4.04.0_ios/build/camomile.1.0.0/camomile-4894-d2c37b.err
### stderr ###
# usage: ar -d [-TLsv] archive file ...
# [...]
# ar -q [-cTLsv] archive file ...
# ar -r [-cuTLsv] archive file ...
# ar -r [-abciuTLsv] position archive file ...
# ar -t [-TLsv] archive [file ...]
# ar -x [-ouTLsv] archive [file ...]
# File "_none_", line 1:
# Error: Error while creating the library Camomile/camomile_yuge.a
# ocamlyacc Camomile/internal/uReStrParser.{ml,mli}
# 4 shift/reduce conflicts.
Camomile started a long time ago and OCaml introduced a lot of new features since then, for instance aliasing modules or extension points. Here is the output
% opam switch
3.11.2 I 3.11.2 Official 3.11.2 release
3.12.1 I 3.12.1 Official 3.12.1 release
4.00.0 I 4.00.0 Official 4.00.0 release
-- -- 4.00.1 Official 4.00.1 release
-- -- 4.01.0 Official 4.01.0 release
-- -- 4.02.0 Official 4.02.0 release
-- -- 4.02.1 Official 4.02.1 release
-- -- 4.02.2 Official 4.02.2 release
4.02.3 C 4.02.3 Official 4.02.3 release
system I system System compiler (4.02.2)
on my system. Which versions do we want to support, i.e. which versions do you need?
Much better than the "packing" that is done now. This will require setting the minimum to 4.02.3
Use Uchar in stdlib, not camomile's own one.
jbuilder will be taking care of this building the docs now
It would be useful to add some CI service to the repository. Since I use Travis CI in most of my OCaml projects and am now used to it, I would propose to implement setup for it. Does anybody wish to use another service? What are the pros/cons against Travis CI?
Seems like cppo is being used not for conditional compilation, but for including module implementations/signatures. The module system should be able to solve these issues adequately.
See http://sourceforge.net/p/camomile/bugs/22/
Patch :
diff --git a/Camomile/internal/unidata.ml b/Camomile/internal/unidata.ml
index b50b46b..3b8a486 100644
--- a/Camomile/internal/unidata.ml
+++ b/Camomile/internal/unidata.ml
@@ -184,7 +184,10 @@ let read_data ?datadir name =
match datadir with
Some d -> d
| None -> Config.datadir in
- Database.read datadir "mar" input_value name
+ try
+ Database.read datadir "mar" input_value name
+ with
+ Not_found -> failwith (Printf.sprintf "Camomile: resource %S not found in database %s" (name^".mar") datadir)
type general_category_type =
[ `Lu (* Letter, Uppercase *)
Camomile should support pinning, i.e. the repository should be pinned in opam.
The upcoming OCaml release (4.06) has now safe-string enabled by default. However camomile isn't compatible with safe-string yet.
I don't have time to make a PR but I'd be happy to give any kind of help in fixing this.
camlp4 was separated from OCaml in 4.02 and is being phased out. It would be great if Camomile did not have a hard dependency on camlp4. The only part of it that depends on camlp4 is tools/camomilelocaledef.ml
, and it is used for the parser syntax.
I think a viable solution is shipping already preprocessed source, e.g. just by doing:
mv tools/camomilelocaledef.ml tools/camomilelocaledef.pp.ml
camlp4o -printers o tools/camomilelocaledef.pp.ml >tools/camomilelocaledef.ml
It is not even much less readable than with camlp4 syntax.
I am using opam 2.0.0 with Ocaml 4.07.1 on OpenBSD 6.3.
When I try to install camonile, opam give me the errors below
> [ERROR] The compilation of camomile failed at "/home/david/.opam/4.07.1/bin/jbuilder build -p camomile -j 1".
>
> #=== ERROR while compiling camomile.1.0.1 =====================================#
> # context 2.0.0 | openbsd/x86_64 | ocaml-base-compiler.4.07.1 | https://opam.ocaml.org#da96e33f
> # path ~/.opam/4.07.1/.opam-switch/build/camomile.1.0.1
> # command ~/.opam/4.07.1/bin/jbuilder build -p camomile -j 1
> # exit-code 1
> # env-file ~/.opam/log/camomile-8942-0a5907.env
> # output-file ~/.opam/log/camomile-8942-0a5907.out
> ### output ###
> # [...]
> # Warning : normalization option is not supported
> # camomilelocaledef Camomile/locales/tt.mar
> # Warning : normalization option is not supported
> # camomilelocaledef Camomile/locales/ur.mar
> # Warning : normalization option is not supported
> # camomilelocaledef Camomile/locales/uz.mar
> # Warning : normalization option is not supported
> # camomilelocaledef Camomile/locales/vi.mar
> # Warning : normalization option is not supported
> # camomilelocaledef Camomile/locales/zh__PINYIN.mar (exit 2)
> # (cd _build/default/Camomile && tools/camomilelocaledef.exe --file locales/zh__PINYIN.txt locales)
> # Fatal error: exception Stack overflow
>
>
>
> <><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
> +- The following actions failed
> | - build camomile 1.0.1
> +-
> - No changes have been performed
>
contents of " ~/.opam/log/camomile-8942-0a5907.out"
File "Camomile/jbuild", line 4, characters 0-96:
4 | (rule (with-stdout-to installConfig.ml
5 | (echo "let share_dir = \"/usr/share/camomile\"")))
Warning: File installConfig.ml is both generated by a rule and present in the source tree.
As a result, the rule is currently ignored, however this will become an error in the future.
To keep the current behavior and get rid of this warning, add a field (fallback) to the rule.
ocamlyacc Camomile/internal/uReStrParser.{ml,mli}
4 shift/reduce conflicts.
camomilelocaledef Camomile/locales/ar.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/az.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/div.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/el.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/fa.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/gu.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/he.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/hi.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/hi__DIRECT.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/ja.mar
Warning : strength option is not supported
camomilelocaledef Camomile/locales/ka.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/kk.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/kn.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/ky.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/mn.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/mr.mar
camomilelocaledef Camomile/locales/mr.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/pa.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/sa.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/syr.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/ta.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/te.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/th.mar
Warning : file loading is not supported.
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/tt.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/ur.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/uz.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/vi.mar
Warning : normalization option is not supported
camomilelocaledef Camomile/locales/zh__PINYIN.mar (exit 2)
(cd _build/default/Camomile && tools/camomilelocaledef.exe --file locales/zh__PINYIN.txt locales)
Fatal error: exception Stack overflow
Support for Unicode 10 or 11 would be better to be postponed.
apparently this is due to some files not being installed by camomile (!). see this issue: ocaml/opam-repository#6305
The latest release 0.8.4 on github is somewhat hidden (camomile.sf.net doesn't have any pointer to github so it is hard to find the new repo) and it doesn't have the same directory structure as 0.8.3 and most importantly there is no configure, one has to autoconf it.
Some of the notes mention that it was originally added by @gildor478 but there's no explanation as to what was the motivation. Are there any use cases where this feature is useful?
Current encoder/decoder dies throwing an exception if they encounter unknown sequences or a character outside of scope of an encoding. Sometime this is not nice and better to mapping errors to the "unknown" character. Doing this securely is a complex task and would not easy.
opam install camomile
was failing for me on Mac OS X 10.9: https://gist.github.com/SethTisue/f99fcb80ffa639951d18
where the key lines are:
# File "internal/uReStrLexer.mll", line 37, characters 0-17:
# Error: Unbound module UReStrParser
someone else obviously hit the same problem: pld-linux/ocaml-camomile@2ca02a0)
I was able to work around it with MAKEFLAGS=-j1 opam install camomile
, but it would be nice if the underlying issue were addressed.
Camomile is currently built by ad-hoc GNU make makefiles. Do we want to change this, as other solutions might be easier to work with?
IMO GNU make has several problems, most notably GNU make is very hard to program, it lacks essential predicates (like “does this target exist”), etc.
Possibilities are probably to use ocamlbuild, oasis or bsdowl, I tend to prefer the latter as generic makefiles provide a clear framework to handle generic builds and still can be marginally customised to handle special cases.
I have no experience with oasis and ocamlbuild but we could also consider them.
I have an error when using Opam 1.2.2 to install Camomile 0.8.5. Running ocaml 3.1.2.
The error is get is:
make: no: Command not found
I also get this error when trying to compile from the 0.8.4 source found in the Wiki. I suspect an error with the makefile but am not sure...
Full details:
$ opam install camomile
The following actions will be performed:
∗ install camomile 0.8.5
=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[camomile] Archive in cache
=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[ERROR] The compilation of camomile failed at "make".
Processing 1/1: [camomile: ocamlfind remove]
#=== ERROR while installing camomile.0.8.5 ====================================#
# opam-version 1.2.2
# os linux
# command make
# path /home/user/.opam/system/build/camomile.0.8.5
# compiler system (3.12.1)
# exit-code 2
# env-file /home/user/.opam/system/build/camomile.0.8.5/camomile-4413-d0f245.env
# stdout-file /home/user/.opam/system/build/camomile.0.8.5/camomile-4413-d0f245.out
# stderr-file /home/user/.opam/system/build/camomile.0.8.5/camomile-4413-d0f245.err
### stdout ###
# [...]
# ocamlc -c -noassert -I internal -I public -I toolslib -I . camomileLibrary.mli
# ocamlc -c -noassert -I internal -I public -I toolslib -I . camomileLibrary.ml
# ocamlc -c -noassert -I internal -I public -I toolslib -I . camomileLibraryDefault.mli
# ocamlc -c -noassert -I internal -I public -I toolslib -I . camomileLibraryDefault.ml
# ocamlc -c -noassert -I internal -I public -I toolslib -I . camomileLibraryDyn.mli
# ocamlc -c -noassert -I internal -I public -I toolslib -I . camomileLibraryDyn.ml
# ocamlc -a -o camomile.cma public/unicodeString.cmo configInt.cmo internal/uReStrParserType.cmo internal/database.cmo camomileDefaultConfig.cmo internal/bitsvect.cmo internal/bytesvect.cmo internal/avlTree.cmo internal/iSet.cmo internal/iMap.cmo internal/tbl31.cmo internal/byte_labeled_dag.cmo internal/xArray.cmo public/oOChannel.cmo public/uChar.cmo public/uSet.cmo public/uMap.cmo public/uCharTbl.cmo public/uText.cmo public/xString.cmo public/subText.cmo public/uLine.cmo public/locale.cmo internal/hangul.cmo internal/unidata.cmo internal/charmap.cmo internal/unimap.cmo public/charEncoding.cmo public/uTF8.cmo public/uTF16.cmo public/uCS4.cmo public/uPervasives.cmo public/uCharInfo.cmo public/uNF.cmo public/uCol.cmo public/caseMap.cmo public/uRe.cmo internal/uReStrParser.cmo internal/uReStrLexer.cmo public/uReStr.cmo internal/stringPrep_data.cmo public/stringPrep.cmo camomileLibrary.cmo camomileLibraryDefault.cmo camomileLibraryDyn.cmo
# ocamlc -a -o camomileLibrary.cma public/unicodeString.cmo configInt.cmo internal/uReStrParserType.cmo internal/database.cmo camomileDefaultConfig.cmo internal/bitsvect.cmo internal/bytesvect.cmo internal/avlTree.cmo internal/iSet.cmo internal/iMap.cmo internal/tbl31.cmo internal/byte_labeled_dag.cmo internal/xArray.cmo public/oOChannel.cmo public/uChar.cmo public/uSet.cmo public/uMap.cmo public/uCharTbl.cmo public/uText.cmo public/xString.cmo public/subText.cmo public/uLine.cmo public/locale.cmo internal/hangul.cmo internal/unidata.cmo internal/charmap.cmo internal/unimap.cmo public/charEncoding.cmo public/uTF8.cmo public/uTF16.cmo public/uCS4.cmo public/uPervasives.cmo public/uCharInfo.cmo public/uNF.cmo public/uCol.cmo public/caseMap.cmo public/uRe.cmo internal/uReStrParser.cmo internal/uReStrLexer.cmo public/uReStr.cmo internal/stringPrep_data.cmo public/stringPrep.cmo camomileLibrary.cmo
# no -c -noassert -I internal -I public -I toolslib -I . -impl public/unicodeString.mli
# rm internal/uReStrLexer.ml
### stderr ###
# make: no: Command not found
# make: *** [public/unicodeString.cmx] Error 127
=-=- Error report -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The following actions failed
∗ install camomile 0.8.5
No changes have been performed
Any help is much appreciated. thanks
+ jbuilder runtest --profile release
File "camomile-test/tester-lib/camomileLibraryTest.ml", line 1, characters 18-38:
1 | module Camomile = CamomileLibrary.Make (struct
^^^^^^^^^^^^^^^^^^^^
Error: Unbound module CamomileLibrary
Generates errors about Pervasives
:
Error (alert deprecated): module Stdlib.Pervasives
Use Stdlib instead.
This is also a part of updating Camomile to the latest Unicode standard. This requires defining new properties and new property values, and taking cares of consistency of all code using UCD. The second thing to do.
Using topkg along with opam-publish to create releases. This has the following advantages:
I can set this up and add some small instructions for how to make a release using topkg.
I'm on a NetBSD system. Camomile 0.8.6 and 0.8.7 won't build under OPAM's OCaml 4.05 or OCaml 4.06 environments. They all result in a SEGV error. Here's the end of the OPAM .err
file:
camomilelocaledef Camomile/locales/zh.mar
camomilelocaledef Camomile/locales/zh_CN.mar
camomilelocaledef Camomile/locales/zh_HK.mar
camomilelocaledef Camomile/locales/zh_MO.mar
camomilelocaledef Camomile/locales/zh_SG.mar
camomilelocaledef Camomile/locales/zh_TW.mar
camomilelocaledef Camomile/locales/zh_TW_STROKE.mar
camomilelocaledef Camomile/locales/zh__PINYIN.mar (got signal SEGV)
(cd _build/default/Camomile && ./tools/camomilelocaledef.exe --file locales/zh__PINYIN.txt locales)
Unfortunately the stack trace isn't too enlightening:
Program terminated with signal 11, Segmentation fault.
#0 0x000000000059eb2b in caml_c_call ()
(gdb)
Any other info I can provide to get this fixed? (I can't install utop
until this builds.)
gasoline, liquidsoup, patoline fail to build using Camomile 1.0.0
Alpine Linux fails to build Camomile 1.0.0
Did we introduce incompatible changes? Wrap the library?
While testing several variants of ocaml for the new upcoming OCaml 4.07 I noticed a weird failure happening inconsistently when trying to build camomile.1.0.1
:
#=== ERROR while compiling camomile.1.0.1 =====================================#
# context 2.0.0~rc | linux/x86_64 | ocaml-variants.4.07.0+beta2+flambda | file:///home/opam/opam-repository
# path ~/.opam/4.07.0+beta2+flambda/.opam-switch/build/camomile.1.0.1
# command ~/.opam/4.07.0+beta2+flambda/bin/jbuilder build -p camomile -j 63
# exit-code 1
# env-file ~/.opam/log/camomile-1-882bca.env
# output-file ~/.opam/log/camomile-1-882bca.out
### output ###
# File "Camomile/jbuild", line 4, characters 0-96:
# Warning: File installConfig.ml is both generated by a rule and present in the source tree.
# As a result, the rule is currently ignored, however this will become an error in the future.
# To keep the current behavior and get rid of this warning, add a field (fallback) to the rule.
# ocamlyacc Camomile/internal/uReStrParser.{ml,mli}
# 4 shift/reduce conflicts.
# parse_specialcasing Camomile/database/special_casing.mar (exit 2)
# (cd _build/default/Camomile && ./tools/parse_specialcasing.exe database unidata/SpecialCasing.txt)
# Fatal error: exception Not_found
Full log here: http://51.145.134.72:8080/4.07.0+beta2+flambda-Oclassic/bad/zed.1.6
I have no idea what is causing this since it only happened once and I couldn't reproduce it afterwards.
cc @mshinwell in case this has something to do with -Oclassic
used here (but I cannot reproduce it, I'll try one more time)
I am trying to get Camomile working on 4.05. So far I discovered the following issues:
There are some patches distributed with the opam
package that are not currently merged upstream, and which I needed to test the 4.05 build (in particular the patch that removes the camlp4o dependency, given that Camlp4 is not currently packaged for 4.05). Should I consider sending these patches here for maintainers to consider them for inclusion? (I have no idea what their content is, but merged them in my testing branch as 8244e5d and 2349f99 .) (After giving them a look, the no-camlp4 patch seems to just change the parser into its post-processed version, so I guess it makes sense to keep distributing separately and not stored the camlp4-generated files in version control, but maybe that could be pointed out in the README or somewhere?)
There is an obviously-safe, minor change that is required by a modification of the type-checker behavior in 4.05, I sent a patch in #20.
In the current 4.05 branch, Bigarray
seems to depend on Unix
because of the change in ocaml/ocaml#997. This breaks the Camomile build, which list .cma
dependencies by hand and lists bigarray.cma
but (of course?) not unix.cma
. I have patched Camomile locally to go further with the build (see my working 4.05-fixes branch which contains all the changes I currently have stacked to test the build), but it is not clear to me that changing Camomile is the right move here: it might be considered a regression in the standard library, and fixed at this level, requiring no Camomile change.
Finally, once all these changes are applied, I see a surprising error when building Camomile, that occurs when building under 4.05 but not under 4.04:
tools/camomilelocaledef.opt --file locales/test_1.txt locales
Fatal error: exception File "toolslib/absCe.ml", line 163, characters 19-25: Assertion failed
Makefile:360: recipe for target 'locales/test_1.mar' failed
make: *** [locales/test_1.mar] Error 2
This may be a bug in the 4.05.0+beta2 compiler or runtime, or at least a change in behavior that breaks Camomile. I don't know/understand the codebase to have an idea of what causes it, but it is worth investigating more.
Part of works which updates Camomile to the latest Unicode. Easiest and maybe the first thing to be done. Just replace charmaps from the latest glibc and look whether it works or not.
This may introduce characters which current Camomile doesn't know, however. Therefore, it would be better to work this on a separate integration branch (which integrates all works related to update Unicode support)
Probably AbsOrd's fault
Hi,
I observe the last commits date back to four years ago and the last release does not compile with the recent versions of OCaml (actually even not so recent versions).
Is it maintained somewhere else or replaced with another library ?
Best,
Many of us would love to see a version 1.0.0 of Camomile! But how can we start to help if we do not have a roadmap? It would be great to prepare one!
Need to investigate what happened after ver 3.2 Need to extract collation rules from LDML files.
Camomile's config file stuff relies on there being a file system that can be written to. This approach doesn't translate to the browser setting. A js_of_ocaml app that uses Camomile as a dependency experiences a runtime error "File does not exist" when it tries to load a config file upon starting.
This error seems to occur when the js_of_ocaml app is built in debug mode with separate compilation, but the config file access is somehow optimized away in release builds.
Unicode standard has introduced extra cases for ancient Greek to handle case mappings (around ver. 7). Also need to investigate the algorithm of the current version (ver. 10)
The library uses camlp4o to parse locale definitions, which is a hinderance to both adoption (see tategakibunko/jingoo#19) as well as future maintenance. The parser should be ported to menhir or some such other parser library to break the dependency with camlp4o.
This is really "FYI" rather than a bug. It may even be a compiler bug. Anyway, when building on ppc64 or ppc64le there's a stack overflow in the final stages of the build:
Running[744]: (cd _build/default/Camomile && ./tools/camomilelocaledef.exe --file locales/zh__PINYIN.txt locales)
Command [744] got signal SEGV:
$ (cd _build/default/Camomile && ./tools/camomilelocaledef.exe --file locales/zh__PINYIN.txt locales)
It happens in the same place on both arches.
I worked around it by doing:
%ifarch %{power64}
ulimit -Hs 65536
ulimit -Ss 65536
%endif
This doesn't happen on the other arches we test with (i686, x86_64, s390, s390x, aarch64, armv7hl).
such as String.set
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.