monterail / elasticsearch-analysis-morfologik Goto Github PK
View Code? Open in Web Editor NEWMorfologik (Polish) Analysis Plugin for ElasticSearch
License: Apache License 2.0
elasticsearch-analysis-morfologik's People
Contributors
Stargazers
Watchers
Forkers
pgaertig ee rn0 episerver slachiewicz kzemek piotrantosik mbuczko durszlak-pl lury zergin ktp-forked-repos purban-itelasticsearch-analysis-morfologik's Issues
Plugin installation doesn't work with latest ElasticSearch 0.90.0 (missing class org.apache.lucene.analysis.ReusableAnalyzerBase)
I've installed morfologik from http://search.maven.org/#search|ga|1|a%3A%22elasticsearch-analysis-morfologik%22 (1.0.0) and cannot create an index, as follow:
{
"settings" : {
"number_of_shards" : 5
},
"mappings" : {
"foto" : {
"_analyzer": "morfologik",
"properties" : {
"info" : {"type": "string", "store": "yes", "index": "analyzed", "index_analyzer": "morfologik", "search_analyzer": "morfologik"},
"title" : {"type": "string", "store": "yes", "index": "analyzed", "index_analyzer": "morfologik", "search_analyzer": "morfologik"},
"path" : {"type": "string", "store": "yes", "index": "analyzed", "index_analyzer": "morfologik", "search_analyzer": "morfologik"}
}
}
}
}[2013-05-01 12:00:51,627][WARN ][cluster.metadata ] [Cobalt Man] [fotos] failed to create
org.elasticsearch.indices.IndexCreationException: [fotos] failed to create index
at org.elasticsearch.indices.InternalIndicesService.createIndex(InternalIndicesService.java:384)
at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:263)
at org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:229)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:95)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NoClassDefFoundError: org/apache/lucene/analysis/ReusableAnalyzerBase
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(Unknown Source)
at java.lang.ClassLoader.defineClass(Unknown Source)
at java.security.SecureClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.access$000(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)
at java.lang.Class.getDeclaredConstructors(Unknown Source)
at org.elasticsearch.common.inject.assistedinject.FactoryProvider.createMethodMapping(FactoryProvider.java:214)
at org.elasticsearch.common.inject.assistedinject.FactoryProvider.newFactory(FactoryProvider.java:151)
at org.elasticsearch.common.inject.assistedinject.FactoryProvider.newFactory(FactoryProvider.java:146)
at org.elasticsearch.index.analysis.AnalysisModule.configure(AnalysisModule.java:407)
at org.elasticsearch.common.inject.AbstractModule.configure(AbstractModule.java:60)
at org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(Elements.java:201)
at org.elasticsearch.common.inject.spi.Elements.getElements(Elements.java:82)
at org.elasticsearch.common.inject.InjectorShell$Builder.build(InjectorShell.java:130)
at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:99)
at org.elasticsearch.common.inject.InjectorImpl.createChildInjector(InjectorImpl.java:129)
at org.elasticsearch.common.inject.ModulesBuilder.createChildInjector(ModulesBuilder.java:66)
at org.elasticsearch.indices.InternalIndicesService.createIndex(InternalIndicesService.java:380)
... 6 more
Caused by: java.lang.ClassNotFoundException: org.apache.lucene.analysis.ReusableAnalyzerBase
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 33 more
Update Morfologik
Hi!
Any plan update to latest Morfologik (2.x) ?
Plugin installation doesn't work with ElasticSearch 0.20.5
This is output of installation attempt with ElasticSearch 0.20.5:
$ bin/plugin -install chytreg/elasticsearch-analysis-morfologik/1.0.0 -verbose
-> Installing chytreg/elasticsearch-analysis-morfologik/1.0.0...
Trying http://download.elasticsearch.org/chytreg/elasticsearch-analysis-morfologik/elasticsearch-analysis-morfologik-1.0.0.zip...
Failed: IOException[Can't get http://download.elasticsearch.org/chytreg/elasticsearch-analysis-morfologik/elasticsearch-analysis-morfologik-1.0.0.zip to /home/pg/bundle/elasticsearch-0.20.5/plugins/analysis-morfologik.zip]; nested: FileNotFoundException[http://download.elasticsearch.org/chytreg/elasticsearch-analysis-morfologik/elasticsearch-analysis-morfologik-1.0.0.zip]; nested: FileNotFoundException[http://download.elasticsearch.org/chytreg/elasticsearch-analysis-morfologik/elasticsearch-analysis-morfologik-1.0.0.zip];
Trying http://search.maven.org/remotecontent?filepath=chytreg/elasticsearch-analysis-morfologik/1.0.0/elasticsearch-analysis-morfologik-1.0.0.zip...
Failed: IOException[Can't get http://search.maven.org/remotecontent?filepath=chytreg/elasticsearch-analysis-morfologik/1.0.0/elasticsearch-analysis-morfologik-1.0.0.zip to
/home/pg/bundle/elasticsearch-0.20.5/plugins/analysis-morfologik.zip]; nested: FileNotFoundException[http://search.maven.org/remotecontent?filepath=chytreg/elasticsearch-analysis-morfologik/1.0.0/elasticsearch-analysis-morfologik-1.0.0.zip]; nested: FileNotFoundException[http://search.maven.org/remotecontent?filepath=chytreg/elasticsearch-analysis-morfologik/1.0.0/elasticsearch-analysis-morfologik-1.0.0.zip];
Trying https://oss.sonatype.org/service/local/repositories/releases/content/chytreg/elasticsearch-analysis-morfologik/1.0.0/elasticsearch-analysis-morfologik-1.0.0.zip...
Failed: IOException[Can't get https://oss.sonatype.org/service/local/repositories/releases/content/chytreg/elasticsearch-analysis-morfologik/1.0.0/elasticsearch-analysis-morfologik-1.0.0.zip to /home/pg/bundle/elasticsearch-0.20.5/plugins/analysis-morfologik.zip]; nested: FileNotFoundException[https://oss.sonatype.org/service/local/repositories/releases/content/chytreg/elasticsearch-analysis-morfologik/1.0.0/elasticsearch-analysis-morfologik-1.0.0.zip]; nested: FileNotFoundException[https://oss.sonatype.org/service/local/repositories/releases/content/chytreg/elasticsearch-analysis-morfologik/1.0.0/elasticsearch-analysis-morfologik-1.0.0.zip];
Trying https://github.com/chytreg/elasticsearch-analysis-morfologik/zipball/v1.0.0... (assuming site plugin)
Failed: IOException[Can't get https://github.com/chytreg/elasticsearch-analysis-morfologik/zipball/v1.0.0 to /home/pg/bundle/elasticsearch-0.20.5/plugins/analysis-morfologik.zip]; nested: FileNotFoundException[https://nodeload.github.com/chytreg/elasticsearch-analysis-morfologik/legacy.zip/v1.0.0]; nested: FileNotFoundException[https://nodeload.github.com/chytreg/elasticsearch-analysis-morfologik/legacy.zip/v1.0.0];
Failed to install chytreg/elasticsearch-analysis-morfologik/1.0.0, reason: failed to download out of all possible locations..., use -verbose to get detailed information
I've also tried without 1.0.0 branch and here is the result of using master:
$ bin/plugin -install chytreg/elasticsearch-analysis-morfologik
-> Installing chytreg/elasticsearch-analysis-morfologik...
Trying https://github.com/chytreg/elasticsearch-analysis-morfologik/zipball/master... (assuming site plugin)
Downloading .......DONE
Plugin installation assumed to be site plugin, but contains source code, aborting installation...
tokens quality - strange
Hi!
I'm using ES 1.3.1 and Morfologik 2.3.1 and there are some strange behaviours observed in analyzis results:
- Duplicate tokens
- Tokens that haven't been mentioned in original strings (ojciec, ocean?! in the following example)
An example:
➜ ~ curl -XGET 'localhost:9200/_analyze?pretty&analyzer=morfologik' -d "Nie zabrakło oczywiście wpadek. Największym zaskoczeniem okazał się dla nas strój Katarzyny Zielińskiej, której ewidentnie o coś chodziło, ale wciąż nie wiemy o co."
{
"tokens" : [ {
"token" : "nie",
"start_offset" : 0,
"end_offset" : 3,
"type" : "<ALPHANUM>",
"position" : 1
}, {
"token" : "on",
"start_offset" : 0,
"end_offset" : 3,
"type" : "<ALPHANUM>",
"position" : 1
}, {
"token" : "zabraknąć",
"start_offset" : 4,
"end_offset" : 12,
"type" : "<ALPHANUM>",
"position" : 2
}, {
"token" : "oczywiście",
"start_offset" : 13,
"end_offset" : 23,
"type" : "<ALPHANUM>",
"position" : 3
}, {
"token" : "wpadka",
"start_offset" : 24,
"end_offset" : 30,
"type" : "<ALPHANUM>",
"position" : 4
}, {
"token" : "duży",
"start_offset" : 32,
"end_offset" : 43,
"type" : "<ALPHANUM>",
"position" : 5
}, {
"token" : "wielki",
"start_offset" : 32,
"end_offset" : 43,
"type" : "<ALPHANUM>",
"position" : 5
}, {
"token" : "zaskoczenie",
"start_offset" : 44,
"end_offset" : 56,
"type" : "<ALPHANUM>",
"position" : 6
}, {
"token" : "zaskoczyć",
"start_offset" : 44,
"end_offset" : 56,
"type" : "<ALPHANUM>",
"position" : 6
}, {
"token" : "okazać",
"start_offset" : 57,
"end_offset" : 63,
"type" : "<ALPHANUM>",
"position" : 7
}, {
"token" : "się",
"start_offset" : 64,
"end_offset" : 67,
"type" : "<ALPHANUM>",
"position" : 8
}, {
"token" : "dla",
"start_offset" : 68,
"end_offset" : 71,
"type" : "<ALPHANUM>",
"position" : 9
}, {
"token" : "my",
"start_offset" : 72,
"end_offset" : 75,
"type" : "<ALPHANUM>",
"position" : 10
}, {
"token" : "strój",
"start_offset" : 76,
"end_offset" : 81,
"type" : "<ALPHANUM>",
"position" : 11
}, {
"token" : "stroić",
"start_offset" : 76,
"end_offset" : 81,
"type" : "<ALPHANUM>",
"position" : 11
}, {
"token" : "Katarzyna",
"start_offset" : 82,
"end_offset" : 91,
"type" : "<ALPHANUM>",
"position" : 12
}, {
"token" : "Zielińska",
"start_offset" : 92,
"end_offset" : 103,
"type" : "<ALPHANUM>",
"position" : 13
}, {
"token" : "który",
"start_offset" : 105,
"end_offset" : 111,
"type" : "<ALPHANUM>",
"position" : 14
}, {
"token" : "ewidentnie",
"start_offset" : 112,
"end_offset" : 122,
"type" : "<ALPHANUM>",
"position" : 15
}, {
"token" : "o",
"start_offset" : 123,
"end_offset" : 124,
"type" : "<ALPHANUM>",
"position" : 16
}, {
"token" : "ocean",
"start_offset" : 123,
"end_offset" : 124,
"type" : "<ALPHANUM>",
"position" : 16
}, {
"token" : "ojciec",
"start_offset" : 123,
"end_offset" : 124,
"type" : "<ALPHANUM>",
"position" : 16
}, {
"token" : "coś",
"start_offset" : 125,
"end_offset" : 128,
"type" : "<ALPHANUM>",
"position" : 17
}, {
"token" : "chodzić",
"start_offset" : 129,
"end_offset" : 137,
"type" : "<ALPHANUM>",
"position" : 18
}, {
"token" : "ale",
"start_offset" : 139,
"end_offset" : 142,
"type" : "<ALPHANUM>",
"position" : 19
}, {
"token" : "wciąż",
"start_offset" : 143,
"end_offset" : 148,
"type" : "<ALPHANUM>",
"position" : 20
}, {
"token" : "nie",
"start_offset" : 149,
"end_offset" : 152,
"type" : "<ALPHANUM>",
"position" : 21
}, {
"token" : "on",
"start_offset" : 149,
"end_offset" : 152,
"type" : "<ALPHANUM>",
"position" : 21
}, {
"token" : "wiedzieć",
"start_offset" : 153,
"end_offset" : 158,
"type" : "<ALPHANUM>",
"position" : 22
}, {
"token" : "o",
"start_offset" : 159,
"end_offset" : 160,
"type" : "<ALPHANUM>",
"position" : 23
}, {
"token" : "ocean",
"start_offset" : 159,
"end_offset" : 160,
"type" : "<ALPHANUM>",
"position" : 23
}, {
"token" : "ojciec",
"start_offset" : 159,
"end_offset" : 160,
"type" : "<ALPHANUM>",
"position" : 23
}, {
"token" : "co",
"start_offset" : 161,
"end_offset" : 163,
"type" : "<ALPHANUM>",
"position" : 24
}, {
"token" : "co",
"start_offset" : 161,
"end_offset" : 163,
"type" : "<ALPHANUM>",
"position" : 24
}, {
"token" : "co",
"start_offset" : 161,
"end_offset" : 163,
"type" : "<ALPHANUM>",
"position" : 24
}, {
"token" : "co",
"start_offset" : 161,
"end_offset" : 163,
"type" : "<ALPHANUM>",
"position" : 24
} ]
}
Same string using stempel analyzer:
➜ ~ curl -XGET 'localhost:9200/_analyze?pretty&analyzer=polish' -d "Nie zabrakło oczywiście wpadek. Największym zaskoczeniem okazał się dla nas strój Katarzyny Zielińskiej, której ewidentnie o coś chodziło, ale wciąż nie wiemy o co."
{
"tokens" : [ {
"token" : "zabraknąć",
"start_offset" : 4,
"end_offset" : 12,
"type" : "<ALPHANUM>",
"position" : 2
}, {
"token" : "oczywista",
"start_offset" : 13,
"end_offset" : 23,
"type" : "<ALPHANUM>",
"position" : 3
}, {
"token" : "wpadek",
"start_offset" : 24,
"end_offset" : 30,
"type" : "<ALPHANUM>",
"position" : 4
}, {
"token" : "największym",
"start_offset" : 32,
"end_offset" : 43,
"type" : "<ALPHANUM>",
"position" : 5
}, {
"token" : "zaskoczyć",
"start_offset" : 44,
"end_offset" : 56,
"type" : "<ALPHANUM>",
"position" : 6
}, {
"token" : "okazać",
"start_offset" : 57,
"end_offset" : 63,
"type" : "<ALPHANUM>",
"position" : 7
}, {
"token" : "stroić",
"start_offset" : 76,
"end_offset" : 81,
"type" : "<ALPHANUM>",
"position" : 11
}, {
"token" : "katarzyna",
"start_offset" : 82,
"end_offset" : 91,
"type" : "<ALPHANUM>",
"position" : 12
}, {
"token" : "zieliński",
"start_offset" : 92,
"end_offset" : 103,
"type" : "<ALPHANUM>",
"position" : 13
}, {
"token" : "ewidentny",
"start_offset" : 112,
"end_offset" : 122,
"type" : "<ALPHANUM>",
"position" : 15
}, {
"token" : "chodzić",
"start_offset" : 129,
"end_offset" : 137,
"type" : "<ALPHANUM>",
"position" : 18
}, {
"token" : "wciązać",
"start_offset" : 143,
"end_offset" : 148,
"type" : "<ALPHANUM>",
"position" : 20
}, {
"token" : "wiemy",
"start_offset" : 153,
"end_offset" : 158,
"type" : "<ALPHANUM>",
"position" : 22
} ]
}
Compatibility with ES 1.1
It doesn't seem to be working with ES 1.1.0. Does it?
I've installed it without problems, but when I
GET /_analyze?analyzer=morfologik
{ "Dwoje kotów" }
I get:
{
"error": "ElasticsearchIllegalArgumentException[failed to find analyzer [morfologik]]",
"status": 400
}
Am I doing something wrong? I'm very new to ES - only beginning to learn.
Binary distribution ships buggy org.apache.lucene.analysis.morfologik.MorfologikFilter
The old MorfologikFilter didn't ignore keywords properly, making the keyword_repeat filter useless.
Adding Lucene's 4.10.4 jar to my elasticsearch 1.7 installation fixed the issue for me.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.