carrot2 / elasticsearch-carrot2 Goto Github PK

Compiled plugin from master, and deployed to ES 1.0 cluster and getting the following error upon startup:

10:23:20 elasticsearch.1 | [2014-02-17 10:23:20,791][INFO ][node                     ] [Chi Demon] version[1.0.0], pid[63363], build[a46900e/2014-02-12T16:18:34Z]
10:23:20 elasticsearch.1 | [2014-02-17 10:23:20,791][INFO ][node                     ] [Chi Demon] initializing ...
10:23:20 elasticsearch.1 | [2014-02-17 10:23:20,804][INFO ][plugins                  ] [Chi Demon] loaded [clustering-carrot2], sites [carrot2]
10:23:21 elasticsearch.1 | {1.0.0}: Initialization Failed ...
10:23:21 elasticsearch.1 | - ExecutionError[java.lang.NoClassDefFoundError: org/elasticsearch/ElasticSearchException]
10:23:21 elasticsearch.1 |  NoClassDefFoundError[org/elasticsearch/ElasticSearchException]
10:23:21 elasticsearch.1 |      ClassNotFoundException[org.elasticsearch.ElasticSearchException]
10:23:21 elasticsearch.1 | exited with code 3

Right now I can't install carrot2 to ES1.4.

The following error appeared:

1. NoSuchMethodError[org.elasticsearch.rest.BaseRestHandler.(Lorg/elasticsearch/common/settings/Settings;Lorg/elasticsearch/client/Client;)V]2) NoSuchMethodError[org.elasticsearch.action.support.TransportAction.(Lorg/elasticsearch/common/settings/Settings;Ljava/lang/String;Lorg/elasticsearch/threadpool/ThreadPool;)V]

I have followed exactly same steps as mentioned to configure carrot2 plugin with my existing elasticsearch(ES) repository. I do not see my existing ES repo data into plugin. Do i need any more configuration steps ?

I only see the page where sample indexing page is visible

changed pom.xml to 2.4.0, and ran mvn clean package

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.3:testCompile (default-testCompile) on project elasticsearch-carrot2: Compilation failure
[ERROR] /home/ec2-user/elasticsearch-carrot2/src/test/java/org/carrot2/elasticsearch/SampleIndexTestCase.java:[109,53] error: cannot find symbol [ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.3:testCompile (default-testCompile) on project elasticsearch-carrot2: Compilation failure
/home/ec2-user/elasticsearch-carrot2/src/test/java/org/carrot2/elasticsearch/SampleIndexTestCase.java:[109,53] error: cannot find symbol

        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
/home/ec2-user/elasticsearch-carrot2/src/test/java/org/carrot2/elasticsearch/SampleIndexTestCase.java:[109,53] error: cannot find symbol

        at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:911)
        at org.apache.maven.plugin.compiler.TestCompilerMojo.execute(TestCompilerMojo.java:153)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
        ... 20 more
[ERROR]

Hi,

I just need to set cluster Size. I could not find a possible way to set cluster size with elastic search queries. I tried attributes but I guess i could not find correct field match. Is that possible to set cluster size ?

I am using a query like below;

GET MyIndex/_search_with_clusters
{
"field_mapping": {
"title": [
"fields.articleText.originalHeadline"
],
"content": [
"fields.articleText.originalSummary"
]
},
"search_request": {
"fields": [
"articleText.originalHeadline",
"articleText.originalSummary"
],
"size":100
},
"query_hint": "",
"algorithm": "kmeans"
}

Right now the log says: ERROR: Plugin [elasticsearch-carrot2] is incompatible with Elasticsearch [2.2.0]. Was designed for version [2.1.1]

Is there a schedule to update the plugin to ES 2.1?

ERROR: Plugin [elasticsearch-carrot2] is incompatible with Elasticsearch [2.1.0]. Was designed for version [2.0.0]

I hope this finds you and Stanislaw doing well.

Hi,

I've noticed plugin doesn't work with the latest version of ES - 1.2.0.

I'm getting this error when trying to start ES:

{1.2.0}: Startup Failed ...
- NoClassDefFoundError[org/elasticsearch/rest/XContentThrowableRestResponse]
        ClassNotFoundException[org.elasticsearch.rest.XContentThrowableRestResponse]

Is there a quick fix for this one?

Looks like the provided url in the Usage Guide must be: http://localhost:9200/_plugin/carrot2/ instead of http://localhost:9200/_plugin/elasticsearch-carrot2/

This is required by Lingo3G, eases the installation too.

Even though it's in the classpath and the license is loaded.
http://pastebin.com/VUicGhr2

elasticsearch    | [2016-09-30 00:28:33,708][DEBUG][bootstrap                ] java.home: /usr/lib/jvm/java-8-openjdk-amd64/jre
elasticsearch    | [2016-09-30 00:28:33,709][DEBUG][bootstrap                ] examining jar: /usr/share/elasticsearch/lib/elasticsearch-2.4.0.jar
elasticsearch    | [2016-09-30 00:28:33,714][DEBUG][bootstrap                ] examining jar: /usr/share/elasticsearch/lib/lingo3g-1.13.0.jar
elasticsearch    | [2016-09-30 00:28:33,715][DEBUG][bootstrap                ] examining jar: /usr/share/elasticsearch/lib/lucene-core-5.5.2.jar
elasticsearch    | [2016-09-30 00:28:33,717][DEBUG][bootstrap                ] examining jar: /usr/share/elasticsearch/lib/compiler-0.8.13.jar
elasticsearch    | [2016-09-30 00:28:33,717][DEBUG][bootstrap                ] examining jar: /usr/share/elasticsearch/lib/lucene-backward-codecs-5.5.2.jar
elasticsearch    | [2016-09-30 00:28:33,718][DEBUG][bootstrap                ] examining jar: /usr/share/elasticsearch/lib/lucene-misc-5.5.2.jar

...

elasticsearch    | [2016-09-30 00:28:36,743][DEBUG][plugin.carrot2           ] [Yandroth] Algorithm not available on classpath: lingo3g
elasticsearch    | [2016-09-30 00:28:36,743][INFO ][plugin.carrot2           ] [Yandroth] Available clustering components: lingo, stc, kmeans, byurl
elasticsearch    | [2016-09-30 00:28:36,743][INFO ][plugin.carrot2           ] [Yandroth] Unavailable clustering components: lingo3g
elasticsearch    | [2016-09-30 00:28:36,744][INFO ][plugin.carrot2           ] [Yandroth] Lexical resources dir: /usr/share/elasticsearch/config/elasticsearch-carrot2/resources
elasticsearch    | [2016-09-30 00:28:36,750][DEBUG][plugin.carrot2           ] [Yandroth] Lingo3G license location scan: /usr/share/elasticsearch/config/elasticsearch-carrot2/license.xml (found).
elasticsearch    | [2016-09-30 00:28:36,752][DEBUG][plugin.carrot2           ] [Yandroth] Lingo3G license location scan: /usr/share/elasticsearch/config/elasticsearch-carrot2/.license.xml (not found).
elasticsearch    | [2016-09-30 00:28:36,753][DEBUG][plugin.carrot2           ] [Yandroth] Lingo3G license location scan: /usr/share/elasticsearch/config/license.xml (not found).
elasticsearch    | [2016-09-30 00:28:36,755][DEBUG][plugin.carrot2           ] [Yandroth] Lingo3G license location scan: /usr/share/elasticsearch/config/.license.xml (not found).

I am using the following software versions

• docker v1.9.1
• elasticsearch v2.1.1
• elasticsearch-carrot2 v2.1.1

I am trying to create a docker container running elasticsearch with the carrot2 plugin. Assuming you have docker installed, the following is what I do.

• mkdir /var/data/es
• docker pull elasticsearch:2.1.1
• docker run -d -v /var/data/es:/usr/share/elasticsearch/data --name=es -p 9200:9200 -p 9300:9300 elasticsearch:2.1.1 elasticsearch -Des.cluster.name=elasticsearch
• docker exec es plugin install org.carrot2/elasticsearch-carrot2/2.1.1
• docker restart es

The docker container es dies. Then I type in docker logs -f es and the following is what I see.

Exception in thread "main" ElasticsearchException[Could not start Carrot2 controller.]; nested: AccessControlException[access denied ("java.io.FilePermission" "src/main/config" "read")];
Likely root cause: java.security.AccessControlException: access denied ("java.io.FilePermission" "src/main/config" "read")
        at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
        at java.security.AccessController.checkPermission(AccessController.java:884)
        at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
        at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
        at sun.nio.fs.UnixPath.checkRead(UnixPath.java:795)
        at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:49)
        at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
        at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
        at java.nio.file.Files.readAttributes(Files.java:1737)
        at java.nio.file.Files.isDirectory(Files.java:2192)
        at org.carrot2.elasticsearch.ControllerSingleton.doStart(ControllerSingleton.java:66)
        at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:68)
        at org.elasticsearch.node.Node.start(Node.java:242)
        at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:221)
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:287)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Refer to the log for complete error details.

I then take a look at ControllerSingleton at https://github.com/carrot2/elasticsearch-carrot2/blob/master/src/main/java/org/carrot2/elasticsearch/ControllerSingleton.java#L66. Why is this clazz looking for src/main/config ?

The directory src/main/config does not exists in the docker container, as verified by, docker exec es ls -l.

I used to work with elasticsearch v1.5.2 and elasticsearch-carrot2 v1.8.0, and creating the docker container and installing the carrot2 plugin the way I described above worked. Any ideas?

The plugin ships with morfologik 2.0.1, it should be compatible with C2's (2.1.0).

Hi,
When do you plan to support 2.3.5?

Thanks.

Any insight to how to solve this (if I forgot to set anything) would be great! Cheers!

I am unable to use the 2.4.0.1 plugin with Elasticsearch. I pretty much copied the code from a previous installation that I had. The exception is thrown on action = 'clustering/cluster' and the proxy is null on the following line..

I have attached a debug frame on line TransportProxyClient@55

Caused by: java.lang.NullPointerException
    at org.elasticsearch.client.transport.support.TransportProxyClient$1.doWithNode(TransportProxyClient.java:58)
    at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:212)
    at org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:55)
    at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:288)
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359)
    at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:86)
    at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:56)
    at itonics.cluster.functions.LingoClusterFunction.getClusters(LingoClusterFunction.java:48)
    at itonics.cluster.DefaultClusterService.cluster(DefaultClusterService.java:20)
    at itonics.document.DocumentDriver.cluster(DocumentDriver.java:108)
    at itonics.document.DocumentDriver.doStuff(DocumentDriver.java:98)
    at itonics.document.DocumentDriver.run(DocumentDriver.java:77)
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800)
    ... 10 more

The exception is thrown on the execute() below.

final String[] indicies = {"test_index"};
final String[] types = {"test_type"};

final ClusteringAction.ClusteringActionRequestBuilder clusteringActionRequestBuilder = new ClusteringAction.ClusteringActionRequestBuilder(client)
        .setQueryHint("") // TODO figure out why this is needed
        .setAlgorithm(clusterRequest.getAlgorithm()) // default is defined in the module.properties for document
        .setSearchRequest(client.prepareSearch(indicies).setTypes(types))
        .addFieldMapping("title", LogicalField.TITLE)
        .addFieldMapping("content", LogicalField.CONTENT)
        .setMaxHits(0); // we don't need to return the hits, just the clusters..

try {
    final ClusteringAction.ClusteringActionResponse clusteringActionResponse = clusteringActionRequestBuilder
            .execute()
            .actionGet();

    return transform(clusteringActionResponse.getDocumentGroups());
} catch (NoNodeAvailableException e) {
    throw new ClusterException("couldn't connect to clustering engine.  Please make sure the document store is running and connected properly", e);
}

UPDATE:
Using the Oracle JVM..
java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

Hello everybody, here are some log excerpt from ES:

[2013-10-16 10:36:43,867][INFO ][lingo3g.cc ] Lexical resources reloaded.
[2013-10-16 10:38:27,315][DEBUG][action.search.type ] [Blevins, Sally] [108] Failed to execute fetch phase
java.lang.RuntimeException: cannot invoke method: substring
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.MethodAccessor.getValue(MethodAccessor.java:63)
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.MapAccessor.getValue(MapAccessor.java:39)
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.VariableAccessor.getValue(VariableAccessor.java:37)
at org.elasticsearch.common.mvel2.optimizers.dynamic.DynamicGetAccessor.getValue(DynamicGetAccessor.java:73)
at org.elasticsearch.common.mvel2.ast.ASTNode.getReducedValueAccelerated(ASTNode.java:108)
at org.elasticsearch.common.mvel2.MVELRuntime.execute(MVELRuntime.java:86)
at org.elasticsearch.common.mvel2.compiler.CompiledExpression.getDirectValue(CompiledExpression.java:123)
at org.elasticsearch.common.mvel2.compiler.CompiledExpression.getValue(CompiledExpression.java:119)
at org.elasticsearch.script.mvel.MvelScriptEngineService$MvelSearchScript.run(MvelScriptEngineService.java:192)
at org.elasticsearch.search.fetch.script.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:75)
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:197)
at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:414)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:406)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.executeFetch(TransportSearchQueryThenFetchAction.java:150)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction$2.run(TransportSearchQueryThenFetchAction.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.RuntimeException: cannot invoke method: min
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.MethodAccessor.getValue(MethodAccessor.java:63)
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.StaticReferenceAccessor.getValue(StaticReferenceAccessor.java:31)
at org.elasticsearch.common.mvel2.optimizers.dynamic.DynamicGetAccessor.getValue(DynamicGetAccessor.java:73)
at org.elasticsearch.common.mvel2.ast.ASTNode.getReducedValueAccelerated(ASTNode.java:108)
at org.elasticsearch.common.mvel2.compiler.ExecutableAccessor.getValue(ExecutableAccessor.java:42)
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.MethodAccessor.executeAll(MethodAccessor.java:140)
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.MethodAccessor.getValue(MethodAccessor.java:48)
... 17 more
Caused by: java.lang.RuntimeException: cannot invoke method: length
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.MethodAccessor.getValue(MethodAccessor.java:63)
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.MapAccessor.getValue(MapAccessor.java:39)
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.VariableAccessor.getValue(VariableAccessor.java:37)
at org.elasticsearch.common.mvel2.optimizers.dynamic.DynamicGetAccessor.getValue(DynamicGetAccessor.java:73)
at org.elasticsearch.common.mvel2.ast.ASTNode.getReducedValueAccelerated(ASTNode.java:108)
at org.elasticsearch.common.mvel2.compiler.ExecutableAccessor.getValue(ExecutableAccessor.java:42)
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.MethodAccessor.executeAll(MethodAccessor.java:140)
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.MethodAccessor.getValue(MethodAccessor.java:48)
... 23 more
Caused by: java.lang.NullPointerException
at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.elasticsearch.common.mvel2.optimizers.impl.refl.nodes.MethodAccessor.getValue(MethodAccessor.java:48)
... 30 more

which appears a couple of times, and is then followed by:

[2013-10-16 10:38:37,725][INFO ][lingo3g.cc ] Lexical resources reloaded.
[2013-10-16 10:38:39,028][DEBUG][action.search.type ] [Blevins, Sally] [161] Failed to execute fetch phase
java.lang.NullPointerException
at ASMAccessorImpl_10516961681381912712750.getValue(Unknown Source)
at org.elasticsearch.common.mvel2.optimizers.dynamic.DynamicGetAccessor.getValue(DynamicGetAccessor.java:73)
at org.elasticsearch.common.mvel2.ast.ASTNode.getReducedValueAccelerated(ASTNode.java:108)
at org.elasticsearch.common.mvel2.compiler.ExecutableAccessor.getValue(ExecutableAccessor.java:42)
at ASMAccessorImpl_18419208811381912707400.getValue(Unknown Source)
at org.elasticsearch.common.mvel2.optimizers.dynamic.DynamicGetAccessor.getValue(DynamicGetAccessor.java:73)
at org.elasticsearch.common.mvel2.ast.ASTNode.getReducedValueAccelerated(ASTNode.java:108)
at org.elasticsearch.common.mvel2.compiler.ExecutableAccessor.getValue(ExecutableAccessor.java:42)
at ASMAccessorImpl_13890246471381912707370.getValue(Unknown Source)
at org.elasticsearch.common.mvel2.optimizers.dynamic.DynamicGetAccessor.getValue(DynamicGetAccessor.java:73)
at org.elasticsearch.common.mvel2.ast.ASTNode.getReducedValueAccelerated(ASTNode.java:108)
at org.elasticsearch.common.mvel2.MVELRuntime.execute(MVELRuntime.java:86)
at org.elasticsearch.common.mvel2.compiler.CompiledExpression.getDirectValue(CompiledExpression.java:123)
at org.elasticsearch.common.mvel2.compiler.CompiledExpression.getValue(CompiledExpression.java:119)
at org.elasticsearch.script.mvel.MvelScriptEngineService$MvelSearchScript.run(MvelScriptEngineService.java:192)
at org.elasticsearch.search.fetch.script.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:75)
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:197)
at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:414)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:406)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.executeFetch(TransportSearchQueryThenFetchAction.java:150)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction$2.run(TransportSearchQueryThenFetchAction.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

and there are loads of pages with this previous message

the request sent is:

"search_request" : {
"fields" : [ "title", "language", "postDate", "categories" ],
"query" : {
"match_all" : {}
},
"script_fields" : {
"content" : {
"script" : "if (_source.content==null) return null;_source.content.substring(0,Math.min(200, _source.content.length()))"
}
},
"size" : 3000
},

"query_hint" : "",
"field_mapping" : {
  "title" : [ "fields.title" ],
  "content" : [ "fields.title" ],
  "language" : [ "fields.language" ]
},
"algorithm" : "lingo3g",
"attributes" : {
  "MultilingualClustering.defaultLanguage" : ENGLISH,
  "reload-resources" : true,
  // "language-recognition" : true,
  "cluster-count-base" : clustercountbase,
  "max-word-df" : 0.7,
  "unindexed-word-label-scorer-weight" : 0.7,
  "preferred-label-length" : 2
}

};

I know clusters can be prevented via the carrot2 stopwords configuration files.

Is it possible specify specific clusters to be excluded on a per-query basis?

I'm running ElasticSearch 0.90.5 and version 1.2.1 of the clustering plugin on my local machine and the examples.html file says my ElasticSearch instance cannot be reached. I don't see any configuration or logs for that, is there something I'm missing? I know that version 1.2.0 of the plugin is recommended, but I wouldn't think a newer version would have connectivity issues.

So, I know the clustering supports all the standard query and filter options, but is there a way to get it to return aggregation data for each cluster?

I've tried including aggregations in the search_request, but that only seems to create a single high-level bucket, rather than aggregating within each cluster.

Awesome tool by the way! Thanks!

When I install the plugin I get this:

datasci@ubuntu:~/Desktop/elasticsearch-1.4.0$ bin/plugin --install org.carrot2/elasticsearch-carrot2/1.7.0
-> Installing org.carrot2/elasticsearch-carrot2/1.7.0...
Trying http://download.elasticsearch.org/org.carrot2/elasticsearch-carrot2/elasticsearch-carrot2-1.7.0.zip...
Trying http://search.maven.org/remotecontent?filepath=org/carrot2/elasticsearch-carrot2/1.7.0/elasticsearch-carrot2-1.7.0.zip...
Downloading ...............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................DONE
Installed org.carrot2/elasticsearch-carrot2/1.7.0 into /home/datasci/Desktop/elasticsearch-1.4.0/plugins/carrot2

but then when I restart elasticsearch I get this error:

[2014-11-23 20:46:32,674][INFO ][node ] [Glob] version[1.4.0], pid[7433], build[bc94bd8/2014-11-05T14:26:12Z]
[2014-11-23 20:46:32,675][INFO ][node ] [Glob] initializing ...
[2014-11-23 20:46:32,705][INFO ][plugins ] [Glob] loaded [clustering-carrot2], sites [head, carrot2]
{1.4.0}: Initialization Failed ...

1. NoSuchMethodError[org.elasticsearch.rest.BaseRestHandler.(Lorg/elasticsearch/common/settings/Settings;Lorg/elasticsearch/client/Client;)V]2) NoSuchMethodError[org.elasticsearch.action.support.TransportAction.(Lorg/elasticsearch/common/settings/Settings;Ljava/lang/String;Lorg/elasticsearch/threadpool/ThreadPool;)V]

Using Elasticsearch 2.2.0 and carrot2 when try to clustering documents with "supported" and unsupported language an exception of UnsupportedOperationException is raised.

This error is raised when is executed langCodeWarnings.add(langCodeString) at line 998 of the org.carrot2.elasticsearch.ClusteringAction class. Probably the cause may be associated in the object org.elasticsearch.common.collect.CopyOnWriteHashSet.CopyOnWriteHashSetthat has been made unmodifiable.

In addition, we found that for the language code ISO639-1 for Greek and Czech is respectively el and cs Instead of gr and cz.

So, if the method langCodeWarnings.add(langCodeString) take a languages not "supported" (cs, el or null) an exception is raised instead of to give a warning message logger.warn("Language mapping not a supported ISO639-1 code: {}", langCodeString); of the line 999.

thanks in advantage
best regards

This is much like include_hits parameter, but allows for greater flexibility (for example clustering on a window of 100 search results, but returning only the top 10 to the client).

When do you guys plan to support 2.3.3?

ERROR: Plugin [elasticsearch-carrot2] is incompatible with Elasticsearch [2.3.3]. Was designed for version [2.3.2]

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

I can see slf4j-api-1.6.2.jar in the zip

I tried a bunch of jar's, but i ended up placing slf4j-nop-1.7.21.jar to get rid of the message.

Hello,
I have question if it is possible to map "language" to some constant value instead of document field.

I have documents from one language in separate index and don't want to add this field to each document if it is not necessary.

Thanks,
Zdenek

i m using elasticsearch-carrot2 plugin,
how i can do configuration for stopwords in the carrot2 plugin please?

Hi, where can I configure stopwords and stop phrases? In Carrot there is stopwords.en. Is this supported in the plugin?

My documents are structured like this :

{
  "id": 123,
  "titles": [{
    "name":"headline",
    "content":"..."
  }, {
    "name":"byline",
    "content":"..."
  }],
  "texts": [{
    "name":"lead",
    "content":"..."
  }, {
    "name":"body",
    "content":"..."
  }, {
    "name":"conclusion",
    "content":"..."
  }]
}

I specified the field mapping like this :

"field_mapping": {
  "title": ["_source.titles.content"],
  "content": ["_source.texts.content"]
}

All articles are in the same cluster, because the plugin does not take into account array of properties, only single properties.

How do I configure the plugin to query from an index other than test?

Hi,

When i tried to install carrot with elasticsearch-0.90.10 i've go the empty list of avaliable algorithms.

Digging the logs i've found the following issue:

java.lang.IncompatibleClassChangeError: Found class org.elasticsearch.rest.RestRequest, but interface was expected
at org.carrot2.elasticsearch.ListAlgorithmsAction$RestListAlgorithmsAction.handleRequest(ListAlgorithmsAction.java:233)
at org.elasticsearch.rest.RestController.executeHandler(RestController.java:159)
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:142)
at org.elasticsearch.http.HttpServer.internalDispatchRequest(HttpServer.java:121)
at org.elasticsearch.http.HttpServer$Dispatcher.dispatchRequest(HttpServer.java:83)
at org.elasticsearch.http.netty.NettyHttpServerTransport.dispatchRequest(NettyHttpServerTransport.java:291)
at org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequestHandler.java:43)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.elasticsearch.common.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:145)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
at org.elasticsearch.common.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
at org.elasticsearch.common.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

I have tried to use various versions of plugin or build it from source.

The ClusteringAction.ClusteringActionRequestBuilder has a method

org.carrot2.elasticsearch.ClusteringAction.ClusteringActionRequestBuilder.setSearchRequest

that will accept a SearchRequestBuilder object. We have multi-search queries that are implemented as an

org.elasticsearch.action.search.MultiSearchRequestBuilder

object. Would it be hard to support sending this as an input to the setSearchRequest method?

ES: 1.7.2
C2: 3.10.4

Add declarative-style tests for the clustering action/ endpoint.

I installed the carrot2 plugin in Elasticsearch 2.1.1 on Windows 10 and went to http://localhost:9200/_plugin/elasticsearch-carrot2/.

I see the demo page, but the algorithms drop down is empty.

I then checked the Network tab in Chrome and I noticed that /algorithms returns HTTP 400 Bad Response.

I finally checked the Elasticsearch logs and I saw this:

java.lang.IllegalArgumentException: No feature for name [_algorithms]

Code wasn't suited for hierarchical clustering.

Gradle switch, sigh.
elastic/elasticsearch#15280

Carrot2 and Shield do not work together.
Failed to load resource: the server responded with a status of 403 (Forbidden) http://http://localhost:9200/_algorithms

I'm using admin user credentials to login but it still said it is an error.

{
"error": {
"root_cause": [
{
"type": "security_exception",
"reason": "action [clustering/list] is unauthorized for user [es_admin]"
}
],
"type": "security_exception",
"reason": "action [clustering/list] is unauthorized for user [es_admin]"
},
"status": 403
}

Hello, I've compiled the plugin for elasticsearch 2.3.3. Everything is working but the algorithm lingo3g is not loaded. The license is in the correct place /elasticsearch-2.3.3/config/elasticsearch-carrot2/license.xml

[2016-06-30 19:26:14,811][INFO ][plugin.carrot2 ] [Foggy Nelson] Available clustering components: lingo, stc, kmeans, byurl
[2016-06-30 19:26:14,814][INFO ][plugin.carrot2 ] [Foggy Nelson] Unavailable clustering components: lingo3g

Am I missing something? (Maybe some jar?)

Thanks.

bin/elasticsearch
[2014-07-25 10:01:49,256][INFO ][node ] [Crimson Dynamo] version[1.3.0], pid[72514], build[1265b14/2014-07-23T13:46:36Z]
[2014-07-25 10:01:49,256][INFO ][node ] [Crimson Dynamo] initializing ...
[2014-07-25 10:01:49,270][INFO ][plugins ] [Crimson Dynamo] loaded [clustering-carrot2], sites [carrot2]
[2014-07-25 10:01:49,378][WARN ][plugins ] [Crimson Dynamo] plugin clustering-carrot2, failed to invoke custom onModule method
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.elasticsearch.plugins.PluginsService.processModule(PluginsService.java:198)
at org.elasticsearch.plugins.PluginsModule.processModule(PluginsModule.java:61)
at org.elasticsearch.common.inject.Modules.processModules(Modules.java:64)
at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:58)
at org.elasticsearch.node.internal.InternalNode.(InternalNode.java:192)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:159)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:70)
at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:203)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
Caused by: java.lang.VerifyError: class org.carrot2.elasticsearch.ClusteringAction$RestClusteringAction overrides final method handleRequest.(Lorg/elasticsearch/rest/RestRequest;Lorg/elasticsearch/rest/RestChannel;)V
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:455)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:367)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.carrot2.elasticsearch.ClusteringPlugin.onModule(ClusteringPlugin.java:91)
... 13 more
{1.3.0}: Initialization Failed ...

1. No implementation for java.util.Map<org.elasticsearch.action.GenericAction, org.elasticsearch.action.support.TransportAction> was bound.2) MalformedParameterizedTypeException[null]3) No implementation for org.elasticsearch.action.GenericAction annotated with @org.elasticsearch.common.inject.multibindings.Element(setName=,uniqueId=179) was bound.

Following the directions to install the plugin, and then immediately visit http://localhost:9200/_plugin/carrot2/

results in the following Javascript error when trying to "Search Cluster and Analyze"

Failed to load resource: the server responded with a status of 400 (Bad Request) http://localhost:9200/_algorithms
Uncaught TypeError: Cannot read property 'total' of undefined 

Running ElasticSearch v1.2.1

Hi,

We are using carrot2 plugin for our elasticsearch cluster and for that we have got production license file.
We want to know that is there any curl command to check the validation of the license file like expiry date and other information.
Please help us here

Thanks and regards

Sukanta