nahanil / cjk-unihan

(Kinda crappy) Node.js interface to Unihan database of CJK characters

cjk-unihan

Simple interface to Unihan database for Node.js. Created for use on HanziPal & available on NPM

This library comes with:

• The simple to use node.js module for query Unihan data
• An SQLite3 database populated with data from the Unihan database
• Perl/bash scripts to generate said database, should you wish to update it, etc

Usage

var cjk_unihan = require("cjk-unihan");

// Query a single field for a specified character
cjk_unihan.get("我", "kTotalStrokes", function(err, result){
  console.log("Single field lookup all done:", result);
  // Single field lookup all done: 7
});

// Get all available information about a character
cjk_unihan.get("我", function(err, result){
  console.log("Full lookup all done:", result);
  // Full lookup all done: { character: '我',
  //   SUnicode: '62.3',
  //   kIRGKangXi: '0412.010',
  //   ...
  //   ...
  // }
});

Generating a fresh SQLite3 database

cd parser

# parser/data should contain a file which contains all the concatenated files downloaded from http://www.unicode.org/Public/UCD/latest/
# This file should be sorted by the first column, otherwise shit will break.

perl parse.pl < data/Unihan.sorted.txt > unihan.parsed

# Then we run the shell script to recreate the database file
create_db.sh

After this we should be set to go. Ideally I will wrap this all up in some simple package that will download the latest copy from the web and automate all this fiddling around

TODO

• Tidy up database generation scripts
• Automate download, concatenation, sorting of downloaded unihan txt files
• Tests would be nice. Real ones and such
• Tidy up the format of some data stored (Should currently be a 1:1 dump of the Unihan data)
  • ie, kHanyuPinyin should probably return an array of monosyllabic pronunciations, the rest of this field can be ignored when inserting to database
  • etc etc

Danger, Will Robinson!

It's very early days here. I'm sure there was another module that I used to use for this purpose, but I can't find it for the life of me so here this is. I expect the API calls to remain much the same as I work on this - it's a simple wrapper around a database to 'get' certain information, and doesn't need to do anything more.

The data returned however will be tidied up, change, prodded, poked, tweaked and modified until I am happier with it. If this happens I will increment version numbers accordingly, but I thought I'd add the Buyer Beware sticker before putting this out in the world.

cjk-unihan's Issues

Newer node versions fails to use sqlite3

For reference, node 6.9.1 results into :

cri-ninja@hp-840:~/Documents/font-2-svg-files$ nodejs
> var cjkUnihan = require("cjk-unihan"); cjkUnihan.get("我", function(err, result){ console.log("Full lookup all done:", result); });
Error: Cannot find module '/home/cri-ninja/Documents/font-2-svg-files/node_modules/sqlite3/lib/binding/node-v46-linux-x64/node_sqlite3.node'
    at Function.Module._resolveFilename (module.js:326:15)
    at Function.Module._load (module.js:277:25)
    at Module.require (module.js:354:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/home/cri-ninja/Documents/font-2-svg-files/node_modules/sqlite3/lib/sqlite3.js:4:15)
    at Module._compile (module.js:410:26)
    at Object.Module._extensions..js (module.js:417:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Module.require (module.js:354:17)
> .exit

Examining the said link , I found out I have the /node_modules/sqlite3/lib/binding/node-v46-linux-x64/node_sqlite3.node (46).

I do :

npm uninstall sqlite3         # uninstall sqlite3
nvm use 4.6.1                 # switch to node 4.6.1
nodejs                        # opens js prompt
> var cjkUnihan = require("cjk-unihan"); cjkUnihan.get("我", function(err, result){ console.log("Full lookup all done:", result); });
{ all: 'Known data' }
> Full lookup all done: undefined

If you have an update, please push it, we would appreciate. We want to provide open license fonts to ease Chinese learning.