Coder Social home page Coder Social logo

您好,我在练习"分析网站日志,打造访问地图"项目时遇到了如下错误,麻烦指导下,谢谢! about ideabook HOT 9 CLOSED

phodal avatar phodal commented on August 23, 2024
您好,我在练习"分析网站日志,打造访问地图"项目时遇到了如下错误,麻烦指导下,谢谢!

from ideabook.

Comments (9)

yunzhongshu avatar yunzhongshu commented on August 23, 2024 2

java.lang.ClassNotFoundException: org.apache.commons.httpclient.protocol.ProtocolSocketFactory ---缺少commons-httpclient-3.0.1的jar

from ideabook.

phodal avatar phodal commented on August 23, 2024

看看这个 elastic/elasticsearch-hadoop#586

你有没有加入相应的 jar 包?

from ideabook.

bitian avatar bitian commented on August 23, 2024

jar包我都加入了,安装的pig0.16.0,将elasticsearch-hadoop下面的jar全部放在pig目录下了。

from ideabook.

phodal avatar phodal commented on August 23, 2024

@bitian

按照 http://stackoverflow.com/questions/17973970/how-to-solve-java-lang-noclassdeffounderror 上的解释,仍然是有可能缺少相应的 jar 包。

from ideabook.

bitian avatar bitian commented on August 23, 2024

你好,这个jar包应当放在那里,我下载下来也放到pig目录下了。

from ideabook.

bitian avatar bitian commented on August 23, 2024

我把jar包加错地方了,现在加上没有这个错误了,但是又出现下面的错误:
2017-05-08 07:13:39,172 [LocalJobRunner Map Task Executor #0] INFO org.elasticsearch.hadoop.util.Version - Elasticsearch Hadoop v5.3.2 [d4a72b6494]
2017-05-08 07:13:40,010 [LocalJobRunner Map Task Executor #0] INFO org.apache.commons.httpclient.HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
2017-05-08 07:13:40,012 [LocalJobRunner Map Task Executor #0] INFO org.apache.commons.httpclient.HttpMethodDirector - Retrying request
2017-05-08 07:13:40,018 [LocalJobRunner Map Task Executor #0] INFO org.apache.commons.httpclient.HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
2017-05-08 07:13:40,018 [LocalJobRunner Map Task Executor #0] INFO org.apache.commons.httpclient.HttpMethodDirector - Retrying request
2017-05-08 07:13:40,019 [LocalJobRunner Map Task Executor #0] INFO org.apache.commons.httpclient.HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
2017-05-08 07:13:40,019 [LocalJobRunner Map Task Executor #0] INFO org.apache.commons.httpclient.HttpMethodDirector - Retrying request
2017-05-08 07:13:40,023 [LocalJobRunner Map Task Executor #0] ERROR org.elasticsearch.hadoop.rest.NetworkClient - Node [127.0.0.1:9200] failed (Connection refused (Connection refused)); no other nodes left - aborting...
2017-05-08 07:13:40,051 [Thread-20] INFO org.apache.hadoop.mapred.LocalJobRunner - map task executor complete.
2017-05-08 07:13:40,106 [Thread-20] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local1316418085_0001
java.lang.Exception: java.io.IOException: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549)
Caused by: java.io.IOException: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.StoreFuncDecorator.putNext(StoreFuncDecorator.java:83)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:144)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:97)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:658)
at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:275)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:65)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:250)
at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:546)
at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:173)
at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:149)
at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:192)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.StoreFuncDecorator.putNext(StoreFuncDecorator.java:75)
... 18 more
Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[127.0.0.1:9200]]
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:150)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:461)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:425)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:429)
at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:155)
at org.elasticsearch.hadoop.rest.RestClient.remoteEsVersion(RestClient.java:627)
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:243)
... 23 more
2017-05-08 07:13:40,286 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2017-05-08 07:13:40,287 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_local1316418085_0001 has failed! Stop running all dependent jobs
2017-05-08 07:13:40,292 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2017-05-08 07:13:40,341 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2017-05-08 07:13:40,402 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2017-05-08 07:13:40,411 [main] ERROR org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce job(s) failed!
2017-05-08 07:13:40,438 [main] INFO org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script Statistics:

HadoopVersion PigVersion UserId StartedAt FinishedAt Features
2.8.0 0.16.0 root 2017-05-08 07:13:28 2017-05-08 07:13:40 UNKNOWN

Failed!

Failed Jobs:
JobId Alias Feature Message Outputs
job_local1316418085_0001 A,LOGS_BASE,RAW_LOGS MAP_ONLY Message: Job failed! home/data_program/log,

Input(s):
Failed to read data from "/home/data_program/access2.log"

Output(s):
Failed to produce result in "home/data_program/log"

Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0

Job DAG:
job_local1316418085_0001

2017-05-08 07:13:40,439 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
2017-05-08 07:13:40,506 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job job_local1316418085_0001 failed, hadoop does not return any error message
Details at logfile: /home/data_program/pig_1494252791860.log
2017-05-08 07:13:40,905 [main] INFO org.apache.pig.Main - Pig script completed in 35 seconds and 91 milliseconds (35091 ms)

from ideabook.

phodal avatar phodal commented on August 23, 2024

@bitian 日志上写着:

Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'

from ideabook.

bitian avatar bitian commented on August 23, 2024

谢谢,我解决了这些问题。

from ideabook.

bitian avatar bitian commented on August 23, 2024

您好,最后还遇到一个问题,是关于country字段没有fielddata属性。如下所示:
image
应当怎么设置这个属性值,谢谢。

from ideabook.

Related Issues (6)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.