Flume-问题集锦 about myexp

[neilChenXie]

Input too long

source deserializer max length

[neilChenXie]

JVM Related

Tools

• jconsole
• free for pc

Jconsole

JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.port=60001"
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.authenticate=false"
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.ssl=false"

http://www.linuxidc.com/Linux/2015-02/113420.htm

Flume 调优

Problem

java.lang.OutOfMemoryError: Java heap space

Solve

JAVA_OPTS="-Xms8192m -Xmx8192m -Xss256k -Xmn2g -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit"

http://my.oschina.net/leejun2005/blog/288136

EXP

Based on timestamp official interceptor, 800M memory for an interceptor

// fast with Charset.UTF-8
String ns = new String(event.getBody());

JVM 参数

Xms 最小内存
Xmx 最大内存
Xmn 年轻代大小

http://unixboy.iteye.com/blog/174173/

[neilChenXie]

Very Slow

Cause

1. String new need coding infomation. String body= new String(event.getBody());
2. com.alibaba.fastjson use too much memory

Solve

1. String body= new String(event.getBody(), Charsets.UTF_8);
2. change to jackson or Gson

Ref

• JVM GC Status