edicl / cl-unicode Goto Github PK

View Code? Open in Web Editor NEW

59.0 59.0 24.0 1.48 MB

Portable Unicode library for Common Lisp

Home Page: https://edicl.github.io/cl-unicode/

Common Lisp 99.86% Batchfile 0.14%

cl-unicode's People

Contributors

Stargazers

Watchers

cl-unicode's Issues

Autogeneratd sources build with CLISP can not be loaded by other Lisps:

After I installed a fresh cl-unicode (from the recent quicklisp), the first Lisp I used it with was CLISP.

After this on CCL and SBCL I have the following error:

Read error between positions 188 and 212 in C:/Users/anton/projects/cl-test-grid/quicklisp/dists/quicklisp/software/cl-unicode-0.1.2/methods.lisp.
Unhandled SERIOUS-CONDITION is signaled: There is no package named "CLOS" .

The file methods.lisp contains code like
(|COMMON-LISP|::|IN-PACKAGE| :|CL-UNICODE|)
(|CLOS|::|DEFMETHOD| |CL-UNICODE|::|SCRIPT| ((|CL-UN...
(|CLOS|::|DEFMETHOD| |CL-UNICODE|::|CODE-BLOCK| ((|C...
(|CLOS|::|DEFMETHOD| |CL-UNICODE|::|AGE| ((|CL-UNICO...
(|CLOS|::|DEFMETHOD| |CL-UNICODE|::|GENERAL-CATEGORY...

As you see DEFMETHOD is from a CLOS package.

Make ASDF rebuild the unicode tables whenever they change.

Cannot compile clear in ABCLv1.9.0

Trying to compile cl-unicode with:
Armed Bear Common Lisp 1.9.0
Java 11.0.19 Ubuntu
OpenJDK 64-Bit Server VM

It triggers:

Error loading /home/fermin/.cache/common-lisp/abcl-1.9.0-fasl43-linux-x64/home/fermin/quicklisp/dists/quicklisp/software/cl-unicode-20210228-git/hash-tables.abcl at line 11 (offset 1202352)
#<THREAD "interpreter" {481AD9B9}>: Debugger invoked on condition of type UNBOUND-VARIABLE
  The variable #:LOOP-LIST-1090112 is unbound.
Restarts:
  0: TRY-RECOMPILING               Recompile hash-tables and try loading it again
  1: RETRY                         Retry loading FASL for #<ASDF/LISP-ACTION:CL-SOURCE-FILE "cl-unicode" "hash-tables">.
  2: ACCEPT                        Continue, treating loading FASL for #<ASDF/LISP-ACTION:CL-SOURCE-FILE "cl-unicode" "hash-tables"> as having been successful.
  3: RETRY                         Retry ASDF operation.
  4: CLEAR-CONFIGURATION-AND-RETRY Retry ASDF operation after resetting the configuration.
  5: ABORT                         Give up on "lem-tests"
  6: REGISTER-LOCAL-PROJECTS       Register local projects and try again.

Triggering accept does continue working, but I don't think that's the idea.

Thanks!

Running the tests of 0.1.6 produces `got an unexpected error: There is no property called "Changes_When_Casemapped"`

Does anyknow know what's going on? This is how I run the tests:

	sbcl --non-interactive \
	     --eval '(require "asdf")' \
	     --eval "(push #p\"${wrksrc}/\" asdf:*central-registry*)" \
	     --eval '(asdf:load-system "cl-unicode/test")' \
	     --eval '(uiop:quit (if (cl-unicode-test:run-all-tests)
	                          0 1))'

Cannot use cl-unicode on Allegro mlisp 10.0

Using mlisp 10.0 (patched up to Dec 2. 2016) and loading via quicklisp (patched up to 2016-10-31), I get this error:

cl-user(1): (ql:quickload :cl-unicode)
To load "cl-unicode":
  Load 1 ASDF system:
    cl-unicode
; Loading "cl-unicode"
; While file-compiling #'"lists.lisp" in
#P"/home/fcbr/quicklisp/dists/quicklisp/software/cl-unicode-0.1.5/lists.lisp"
; starting at file character position 141:
Error: Package "CL-UNICODE-NAMES" not found. [file position = 219]
  [condition type: reader-error]

Restart actions (select using :continue):
 0: retry the compilation of /home/fcbr/quicklisp/dists/quicklisp/software/cl-unicode-0.1.5/lists.lisp

unable to install via quicklisp

i'm running SBCL, with slime.

when i call (ql:quickload :cl-unicode) i get an error as follows

no symbol named "*STANDARD-OPTIMIZE-SETTINGS*" in "CL-PPCRE"

if i intern it, i get:

compile-file-error while
compiling #<cl-source-file "cl-unicode/base" "util">

and:

; processing (DECLAIM (INLINE COMPUTE-HANGUL-NAME))
; 
; caught ERROR:
;   READ error during COMPILE-FILE:
;   
;     The variable *STANDARD-OPTIMIZE-SETTINGS* is unbound.
;   
;     (in form starting at line: 219, column: 0, position: 9870)

; compilation aborted after 0:00:00.156

the variable is from cl-ppcre, which is currently loaded.

Value of 'NIL in (THE NIL (PROGN 'NIL)) is NIL, not a NIL.

(ql:quickload :cl-unicode) on SBCL 2.1.0 and 2.0.1 on Linux amd64:

debugger invoked on a SIMPLE-TYPE-ERROR in thread
#<THREAD "main thread" RUNNING {1004A684B3}>:
  Value of 'NIL in (THE NIL (PROGN 'NIL)) is NIL, not a NIL.

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RETRY                        ] Retry
                                     completing load for #<SYSTEM "cl-unicode/build">.
  1: [ACCEPT                       ] Continue, treating
                                     completing load for #<SYSTEM "cl-unicode/build">
                                     as having been successful.
  2:                                 Retry ASDF operation.
  3: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the
                                     configuration.
  4:                                 Retry ASDF operation.
  5:                                 Retry ASDF operation after resetting the
                                     configuration.
  6: [ABORT                        ] Give up on "cl-unicode"
  7:                                 Exit debugger, returning to top level.

(SB-C::%COMPILE-TIME-TYPE-ERROR (NIL) NIL #<unused argument> ((QUOTE NIL)) "(THE NIL (PROGN 'NIL))" NIL)

Backtrace:

0: (SB-C::%COMPILE-TIME-TYPE-ERROR (NIL) NIL #<unused argument> ((QUOTE NIL)) "(THE NIL (PROGN 'NIL))" NIL)
1: ((LAMBDA (SB-PCL::|.P0.| SB-PCL::|.P1.| SB-PCL::|.P2.| SB-PCL::|.P3.| SB-PCL::|.P4.| SB-PCL::|.P5.| SB-PCL::|.P6.| SB-PCL::|.P7.| SB-PCL::|.P8.| SB-PCL::|.P9.| SB-PCL::|.P10.| SB-PCL::|.P11.| SB-PCL::|.P12.|)) #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument>)
2: ((FLET CL-UNICODE::THUNK :IN CL-UNICODE::READ-CHARACTER-DATA) 0)
3: (CL-UNICODE::READ-CHARACTER-DATA)
4: (CL-UNICODE::FILL-DATABASE)
5: (CL-UNICODE::BUILD-DATA-STRUCTURES)
6: (CL-UNICODE::CREATE-SOURCE-FILES)
7: (UIOP/PACKAGE:SYMBOL-CALL :CL-UNICODE #:CREATE-SOURCE-FILES)
8: ((:METHOD ASDF/ACTION:PERFORM (ASDF/LISP-ACTION:LOAD-OP (EQL #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">))) #<unused argument> #<unused argument>) [fast-method]
9: ((SB-PCL::EMF ASDF/ACTION:PERFORM) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">)
10: ((FLET CALL-NEXT-METHOD :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
11: ((LAMBDA NIL :IN ASDF/ACTION:PERFORM))
12: ((LAMBDA NIL :IN ASDF/ACTION:CALL-WHILE-VISITING-ACTION))
13: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/ACTION:CALL-WHILE-VISITING-ACTION) {1007D86DAB}> :OVERRIDE NIL :KEY NIL :OVERRIDE-CACHE NIL :OVERRIDE-FORCING NIL)
14: (ASDF/ACTION:CALL-WHILE-VISITING-ACTION #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build"> #<FUNCTION (LAMBDA NIL :IN ASDF/ACTION:PERFORM) {1007D86D7B}>)
15: ((:METHOD ASDF/ACTION:PERFORM :AROUND (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">) [fast-method]
16: ((SB-PCL::SDFUN-METHOD ASDF/ACTION:PERFORM) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">)
17: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS (T T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">) [fast-method]
18: ((FLET CALL-NEXT-METHOD :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
19: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS :AROUND (T T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode/build">) [fast-method]
20: ((:METHOD ASDF/PLAN:PERFORM-PLAN (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {10044D9A93}>) [fast-method]
21: ((FLET CALL-NEXT-METHOD :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
22: ((FLET "LAMBDA0" :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
23: ((FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
24: ((:METHOD ASDF/PLAN:PERFORM-PLAN :AROUND (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {10044D9A93}>) [fast-method]
25: ((:METHOD ASDF/OPERATE:OPERATE (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode"> :PLAN-CLASS NIL :PLAN-OPTIONS NIL) [fast-method]
26: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode"> :VERBOSE NIL)
27: ((FLET CALL-NEXT-METHOD :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
28: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
29: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {10044D618B}> :OVERRIDE NIL :KEY NIL :OVERRIDE-CACHE NIL :OVERRIDE-FORCING NIL)
30: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "cl-unicode"> :VERBOSE NIL) [fast-method]
31: ((:METHOD ASDF/OPERATE:OPERATE (SYMBOL T)) ASDF/LISP-ACTION:LOAD-OP "cl-unicode" :VERBOSE NIL) [fast-method]
32: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> ASDF/LISP-ACTION:LOAD-OP "cl-unicode" :VERBOSE NIL)
33: ((FLET CALL-NEXT-METHOD :IN "/home/phoe/Projects/Lisp/asdf/build/asdf.lisp"))
34: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
35: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {10044CB82B}> :OVERRIDE NIL :KEY NIL :OVERRIDE-CACHE NIL :OVERRIDE-FORCING NIL)
36: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP "cl-unicode" :VERBOSE NIL) [fast-method]
37: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
38: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {10044CB01B}> :OVERRIDE T :KEY NIL :OVERRIDE-CACHE T :OVERRIDE-FORCING NIL)
39: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
40: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {100400826B}> :OVERRIDE NIL :KEY NIL :OVERRIDE-CACHE NIL :OVERRIDE-FORCING NIL)
41: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP "cl-unicode" :VERBOSE NIL) [fast-method]
42: (ASDF/OPERATE:LOAD-SYSTEM "cl-unicode" :VERBOSE NIL)
43: (QUICKLISP-CLIENT::CALL-WITH-MACROEXPAND-PROGRESS #<FUNCTION (LAMBDA NIL :IN QUICKLISP-CLIENT::APPLY-LOAD-STRATEGY) {1004006BBB}>)
44: (QUICKLISP-CLIENT::AUTOLOAD-SYSTEM-AND-DEPENDENCIES "cl-unicode" :PROMPT NIL)
45: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION (T T)) #<unused argument> #<FUNCTION (FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD) {10033B921B}>) [fast-method]
46: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION :AROUND (QL-IMPL:SBCL T)) #<QL-IMPL:SBCL {1004F34173}> #<FUNCTION (FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD) {10033B921B}>) [fast-method]
47: ((:METHOD QUICKLISP-CLIENT:QUICKLOAD (T)) :CL-UNICODE :PROMPT NIL :SILENT NIL :VERBOSE NIL) [fast-method]
48: (QL-DIST::CALL-WITH-CONSISTENT-DISTS #<FUNCTION (LAMBDA NIL :IN QUICKLISP-CLIENT:QUICKLOAD) {10033B302B}>)
49: (SB-INT:SIMPLE-EVAL-IN-LEXENV (QUICKLISP-CLIENT:QUICKLOAD :CL-UNICODE) #<NULL-LEXENV>)
50: (EVAL (QUICKLISP-CLIENT:QUICKLOAD :CL-UNICODE))
51: (INTERACTIVE-EVAL (QUICKLISP-CLIENT:QUICKLOAD :CL-UNICODE) :EVAL NIL)
52: (SB-IMPL::REPL-FUN NIL)
53: ((LAMBDA NIL :IN SB-IMPL::TOPLEVEL-REPL))
54: (SB-IMPL::%WITH-REBOUND-IO-SYNTAX #<FUNCTION (LAMBDA NIL :IN SB-IMPL::TOPLEVEL-REPL) {10033B2ECB}>)
55: (SB-IMPL::TOPLEVEL-REPL NIL)
56: (SB-IMPL::TOPLEVEL-INIT)
57: ((FLET SB-UNIX::BODY :IN SB-IMPL::START-LISP))
58: ((FLET "WITHOUT-INTERRUPTS-BODY-1" :IN SB-IMPL::START-LISP))
59: (SB-IMPL::START-LISP)

Cannot package CL-UNICODE as a jar

I am linking this issue here because I'm not sure if it is an asdf-jar issue or if it is a problem with that way that cl-unicode is packaged.

armedbear/abcl#371

:description

Would you please consider adding a :description option to your system definition of cl-unicode?

char-info:case-folding-mapping type is nil - ccl fails

when trying to (ql:quickload "cl-unicode") or (asdf:load-system :cl-unicode), clozure cl fails with

The value NIL, derived from the initform NIL, can not be used to set the value of 
the slot CL-UNICODE::CASE-FOLDING-MAPPING in #<CL-UNICODE::CHAR-INFO #x30200268A09D>, 
because it is not of type NIL.
   [Condition of type CCL::BAD-SLOT-TYPE-FROM-INITFORM]

the error is coming from char-info:case-folding-mapping which has the type nil.

the documentation reads

The case folding mapping of the character (as a list of (status, code point)) if explicitly specified

so perhaps (or list null) is a better option than nil? what do you think?

System definition mode-line has wrong package

The system definition mode line specifies a package of CL-USER, however the correct package name is ASDF. On many platforms, asdf takes care of this, but on Genera the mode line is honored and therefore defsystem is not a known symbol. Either the mode line can be patched, or the fully qualified symbol used, e.g. asdf:defsystem.

In default-bidi-class, binary-props* is called on a fixnum value

https://github.com/edicl/cl-unicode/blame/master/build/read.lisp#L238

Should this read (binary-props* char-info)?

In (binary-props* code-point) code-point is a fixnum and binary-props* is an accessor

merge 0.1.3 from edi & send info

no symbol named "STANDARD-OPTIMIZE-SETTINGS" in "CL-PPCRE"

Hi,

(ql:quickload :cl-unicode) results in the error as in the title, the implementation I use is sbcl:

$ sbcl --version
SBCL 1.4.2-1.fc27

Integrating cl-unicode with my Google Summer of Code work

I'm currently working on improving SBCL's Unicode support as part of the Google Summer of Code. Part of my work, which you can find here, involved expanding the SBCL Unicode database and exposing its contents through external APIs, which are in the sb-unicode package and defined in target-unicode.lisp.

Many of the functions that my work provides overlap with those provided by cl-unicode. I am writing to you to discuss whether cl-unicode could become a (partial) wrapper around the SBCL-provided Unicode database when compiled on SBCL to avoid data duplication. I'm willing to add additional data to the internal database, such as block information, to accomplish this goal if you think it's a good idea. One possible issue is that my work stores full case mappings, while yours stores simple mappings.

If you have any other suggestions about how my work could integrate with your library, please let me know.

why CL-UNICODE::CODE-POINTS-TO-UNICODE1-NAMES does not contain entries for ALL code points?

This code should print the entire (code point -> canonical name)
mapping, right ? :

 (with-hash-table-iterator ( it CL-UNICODE::*CODE-POINTS-TO-UNICODE1-NAMES* )
  (loop
   (multiple-value-bind (more i) (it)
    (progn
     (when (not more) (return))
      (format t "~A ~A~%" i (gethash i CL-UNICODE::*CODE-POINTS-TO-UNICODE1-NAMES*))
     )
    )
   )
  )

But that print-out misses some characters, such as \u2248 and \u2249 - where are they ? :
an excerpt from the print-out obtained by running the above code:
...
8682 WHITE UP ARROW FROM BAR
8788 COLON EQUAL
8789 EQUAL COLON
8804 LESS THAN OR EQUAL TO
...
Why isn't it printing \u2248 (8776) : '≈' or \u2449 (8777) : ' ≉' ?
Yet unicode-name resolves them OK :
CL-USER> (CL-UNICODE:unicode-name #\u2248)
-> "ALMOST EQUAL TO"
CL-USER> (CL-UNICODE:unicode-name #\u2249)
-> "NOT ALMOST EQUAL TO"

And they are in Unicode v1 :
CL-USER> (CL-UNICODE:age #\u2248)
-> (1 1)

So that symbol is in unicode v1, so it should be a unicode1 name, and hence in
the hash table ? What am I missing ? Why doesn't the print-out produced
by above code include #\ALMOST_EQUAL_TO ?

Just wondering what the rules for inclusion in that table were,
and if there is a more complete way of printing ALL recognized
code points and names ?

Is cl-unicode somehow checking my locale and deciding which version
of unicode names to include in the table, and omitting some because of version issues ?

It is very easy to print out a unicode table with eg. bash, not so
easy to browse it by symbol name / meaning :-)

Thanks for cl-unicode!
Best Regards,
Jason

Quicklisp symbol-call error.

The error below occurs in LispWorks Personal Edition (version 6.1.1) running on macOs Mojave (version 10.14.1) while loading via quickload.

(ql:quickload "cl-unicode")

Error: Error while trying to load definition for system cl-unicode from pathname .../quicklisp/dists/quicklisp/software/cl-unicode-20180328-git/cl-unicode.asd:
(SYMBOL-CALL :CL-UNICODE (QUOTE #:CREATE-SOURCE-FILES)) does not match (ASDF::O ASDF::C).

Normalize function

For the data processing I do time to time it is quite often really useful to have a 'normalize' function. Such a function converts unicode character to their ASCII 'equivalent' character or string. This is useful as a lot of older systems do not support unicode and to match strings I have to convert the unicode ones to the older ASCII equivalent. The function I use to do this is a simple lookup in a large lookup table, like this:

(defun normalize (string)
           (check-type string string)
           (format nil "~{~A~}"
                   (loop :for el :across string
                         :collect (aref +unicode-lookup-table+
                                        (char-code el)))))

I think such a function could be really useful in a unicode library. Is this something that fits this library and how would you feel about a pull request to add this functionality?

edicl / cl-unicode Goto Github PK

cl-unicode's People

Contributors

Stargazers

Watchers

Forkers

cl-unicode's Issues

Recommend Projects

Recommend Topics

Recommend Org