Coder Social home page Coder Social logo

docker-rocketmq's Issues

java代码中name-server写多少

在centos7上按照楼主的默认配置启动了mq,
在java代码中name-server写成宿主机IP:9876,
测试无法连通mq,报错如下:
org.apache.rocketmq.remoting.exception.RemotingConnectException: connect to <172.20.0.3:10911> failed
Send [3] times, still failed, cost [6011]ms, Topic: test_topic, BrokersSent: [broker-a, broker-a, broker-a]\nSee http://rocketmq.apache.org/docs/faq/ for further details.; nested exception is org.apache.rocketmq.client.exception.MQClientException: Send [3] times, still failed, cost [6011]ms, Topic: test_topic, BrokersSent: [broker-a, broker-a, broker-a]\nSee http://rocketmq.apache.org/docs/faq/ for further details.
控制台http://宿主机IP:8180/#/producer,可正常使用

将broker.conf中的 #brokerIP1=192.168.0.253 这个地方IP改为宿主机IP后,
java代码可正常使用MQ
但是控制台http://宿主机IP:8180/#/producer,会报错连接失败

4.5.2 docker-compose编排的服务无法挂载日志和数据

  - ./rmq/logs:/opt/logs
  - ./rmq/store:/opt/store

这里指定的挂载目录,对应容器里面的目录为空。
rocket启动之后没有指定日志的情况下,是会在home目录下的用户名下生成日志,所以日志是挂载不出来的,store感觉也差不多 是这样

无论怎么设置brokerIP1,Mac下测试都不通过

出错信息如下:

Caused by: org.apache.rocketmq.remoting.exception.RemotingConnectException: connect to host.docker.internal:10911 failed
  • broker.conf

无论设置成局域网IP还是host.docker.internal,都不行

# brokerIP1=192.168.1.2
# brokerIP1=192.168.65.2
brokerIP1=host.docker.internal
  • docker-compose.yml 有两处修改

就了参数:"-n","docker.for.mac.host.internal:9876"

# rmqnamesrv
command: ["sh","mqnamesrv", "-n", "docker.for.mac.host.internal:9876"]

# rmqbroker
command: ["sh","mqbroker","-c","/etc/rocketmq/broker.conf","-n","docker.for.mac.host.internal:9876","autoCreateTopicEnable=true"]

我的测试代码:

export NAMESRV_ADDR=0.0.0.0:9876
cd rocketmq-all-4.7.0-bin-release/
sh bin/tools.sh org.apache.rocketmq.example.quickstart.Producer

我的应用服务不会和RocketMQ在统一网络rmq下,也可能就不是容器启动。
测试的机器是Mac,局域网内192.168.1.2,所以没有公网IP。

org.apache.rocketmq.client.exception.MQClientException: No route info of this topic: TopicTest

执行:sh ./tools.sh org.apache.rocketmq.example.quickstart.Producer 报错
org.apache.rocketmq.client.exception.MQClientException: No route info of this topic: TopicTest
See http://rocketmq.apache.org/docs/faq/ for further details.
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:685)
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1343)
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1289)
at org.apache.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:325)
at org.apache.rocketmq.example.quickstart.Producer.main(Producer.java:67)

配置已经开了:autoCreateTopicEnable=true

4.6.1以后的镜像都有问题

java.io.FileNotFoundException: /etc/rocketmq/broker.conf (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at java.io.FileInputStream.(FileInputStream.java:93)
at org.apache.rocketmq.broker.BrokerStartup.createBrokerController(BrokerStartup.java:128)
at org.apache.rocketmq.broker.BrokerStartup.main(BrokerStartup.java:58)

没测试过的垃圾就不要往上传了,浪费时间给你找问题

on centos7.4 docker RocketMQ , java.io.IOException: Connection reset by peer

Environment:
1、centos7.4
2、foxiswho/rocketmq:server
3、foxiswho/rocketmq:broker

RocketMQ ErrorMsg:
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.8.0_222]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.8.0_222]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.8.0_222]
at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[na:1.8.0_222]
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[na:1.8.0_222]
at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:221) ~[netty-all-4.0.42.Final.jar:4.0.42.Final]
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:899) ~[netty-all-4.0.42.Final.jar:4.0.42.Final]
at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:275) ~[netty-all-4.0.42.Final.jar:4.0.42.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119) ~[netty-all-4.0.42.Final.jar:4.0.42.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:652) ~[netty-all-4.0.42.Final.jar:4.0.42.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:575) ~[netty-all-4.0.42.Final.jar:4.0.42.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:489) ~[netty-all-4.0.42.Final.jar:4.0.42.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:451) ~[netty-all-4.0.42.Final.jar:4.0.42.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140) [netty-all-4.0.42.Final.jar:4.0.42.Final]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_222]

To deploy an mq with a public network address, what parameters need to be changed

cat >/data0/rocketmq/rmq/brokerconf/broker.conf
brokerClusterName=DefaultCluster
brokerName=broker-a
brokerId=0
listenPort=10911
namesrvAddr=rmqnamesrv:9876
brokerIP1=192.168.0.253
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
maxMessageSize=65536
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH

java.io.FileNotFoundException: /etc/rocketmq/broker.conf (Is a directory)问题解决办法

当执行./start.sh时候,broker一直启动失败 报错,如下:
java.io.FileNotFoundException: /etc/rocketmq/broker.conf (Is a directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at java.io.FileInputStream.(FileInputStream.java:93)
at org.apache.rocketmq.broker.BrokerStartup.createBrokerController(BrokerStartup.java:128)
at org.apache.rocketmq.broker.BrokerStartup.main(BrokerStartup.java:58)

原因是 要修改一下docker-compose的内容:
image
修改之后 调用./start.sh 然后就会看到启动成功,如下

image

亲测有效

4.5.2

-server -Xms1994M -Xmx1994M -Xmn400M -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=1994M -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/ext:/home/rocketmq/rocketmq-4.5.2/bin/../lib -server -Xms128m -Xmx128m -Xmn128m -Duser.home=/opt -cp .:/home/rocketmq/rocketmq-4.5.2/bin/../conf:

java.io.FileNotFoundException: /etc/rocketmq/broker.conf (No such file or directory)

at java.io.FileInputStream.open0(Native Method)

at java.io.FileInputStream.open(FileInputStream.java:195)

at java.io.FileInputStream.<init>(FileInputStream.java:138)

at java.io.FileInputStream.<init>(FileInputStream.java:93)

at org.apache.rocketmq.broker.BrokerStartup.createBrokerController(BrokerStartup.java:128)

at org.apache.rocketmq.broker.BrokerStartup.main(BrokerStartup.java:58)

ACL配置文件改哪个

版本4.5.2

ACL配置文件改哪个,nameser和broker都有plain_acl.yml。请问改哪个或者映射到哪个路径?

4.8.0启动broker失败。

version: '3.5'

services:
  rmqbroker:
    image: foxiswho/rocketmq:4.8.0
    container_name: rmqbroker
    ports:
      - 10909:10909
      - 10911:10911
      - 10912:10912
    volumes:
      - ./logs:/home/rocketmq/logs
      - ./store:/home/rocketmq/store
      - ./conf:/home/rocketmq/rocketmq-4.8.0/conf
    environment:
      JAVA_OPT_EXT: "-Duser.home=/home/rocketmq -Xms512M -Xmx512M -Xmn128m"
    command: ["sh","mqbroker","-c","/home/rocketmq/rocketmq-4.8.0/conf/broker.conf","-n","192.168.2.67:9876","autoCreateTopicEnable=true"]

在启动时报错

rmqbroker    | ======================================
rmqbroker    | -server -Xms2934M -Xmx2934M -Xmn733M -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=2934M -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-0.el7_9.x86_64/jre/lib/ext:/home/rocketmq/rocketmq-4.8.0/bin/../lib -Duser.home=/home/rocketmq -Xms512M -Xmx512M -Xmn128m -cp .:/home/rocketmq/rocketmq-4.8.0/bin/../conf:
rmqbroker    | ch.qos.logback.core.joran.spi.JoranException: Could not open [/home/rocketmq/rocketmq-4.8.0/conf/logback_broker.xml].
rmqbroker    | 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:80)
rmqbroker    | 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:68)
rmqbroker    | 	at org.apache.rocketmq.broker.BrokerStartup.createBrokerController(BrokerStartup.java:190)
rmqbroker    | 	at org.apache.rocketmq.broker.BrokerStartup.main(BrokerStartup.java:58)
rmqbroker    | Caused by: java.io.FileNotFoundException: /home/rocketmq/rocketmq-4.8.0/conf/logback_broker.xml (No such file or directory)
rmqbroker    | 	at java.io.FileInputStream.open0(Native Method)
rmqbroker    | 	at java.io.FileInputStream.open(FileInputStream.java:195)
rmqbroker    | 	at java.io.FileInputStream.<init>(FileInputStream.java:138)
rmqbroker    | 	at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:75)
rmqbroker    | 	... 3 more
rmqbroker exited with code 255

显示conf文件夹下没有logback相关文件,请问这个如何解决

Initial heap size set to a larger value than the maximum heap size

b7e32aac2a5f        foxiswho/rocketmq:broker-4.6.1        "sh mqbroker -c /etc…"   17 seconds ago      Exited (1) 15 seconds ago                                                                                       rmqbroker
9e5345a98db3        foxiswho/rocketmq:server-4.6.1        "/bin/sh -c 'cd ${RO…"   18 seconds ago      Exited (1) 16 seconds ago                                                                                       rmqnamesrv
$ docker logs rmqbroker
======================================
-server -Xms487M -Xmx487M -Xmn121M -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=487M -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/lib/ext:/home/rocketmq/rocketmq-4.6.1/bin/../lib -Duser.home=/opt -server -Xms512m -Xms512m -Xmn128m -cp .:/home/rocketmq/rocketmq-4.6.1/bin/../conf:
Error occurred during initialization of VM
Initial heap size set to a larger value than the maximum heap size

$ docker logs rmqnamesrv
OpenJDK 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
OpenJDK 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
Error occurred during initialization of VM
Initial heap size set to a larger value than the maximum heap size

无法启动broker,也没有日志

我直接用你的docker-compose.yaml按照你的说明直接启动部署是正常的,但我要配置日志路径和我自己的conf映射时,配置完之后,日志也没有,broker也启动不起来,我去看你的docker构建指令,logs,store确实都在home目录下,而且你默认的conf也没有改动这些,但是用你默认的sh部署却都在opt目录下,然后我也没改opt,只改了我宿主机的目录路径,然后就凉了,我人都傻了,搞不懂
这是我基于你的yaml改动的结果
version: '3.5'

services:
  rmqnamesrv-a:
    image: foxiswho/rocketmq:4.7.0
    #    image: registry.cn-hangzhou.aliyuncs.com/foxiswho/rocketmq:4.7.0
    container_name: rmqnamesrv-a
    ports:
      - 9876:9876
    volumes:
      - /home/matcloud/DATA/rocketmq/nameserver-a/logs:/opt/logs
      - /home/matcloud/DATA/rocketmq/nameserver-a/store:/opt/store
    environment:
      JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"
    command: ["sh","mqnamesrv"]
    networks:
      rmq:
        aliases:
          - rmqnamesrv-a
  rmqbroker:
    image: foxiswho/rocketmq:4.7.0
    #    image: registry.cn-hangzhou.aliyuncs.com/foxiswho/rocketmq:4.7.0
    container_name: rmqbroker
    ports:
      - 10909:10909
      - 10911:10911
    volumes:
      - /home/matcloud/DATA/rocketmq/broker-b/logs:/opt/logs
      - /home/matcloud/DATA/rocketmq/broker-b/store:/opt/store
      - /home/matcloud/DATA/rocketmq/broker-b/conf/broker.conf:/etc/rocketmq/broker.conf
    environment:
      JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"
    command: ["sh","mqbroker","-c","/etc/rocketmq/broker.conf","-n","rmqnamesrv:9876","autoCreateTopicEnable=true"]
    depends_on:
      - rmqnamesrv-a
    networks:
      rmq:
        aliases:
          - rmqbroker

  rmqconsole:
    image: styletang/rocketmq-console-ng
    container_name: rmqconsole
    ports:
      - 8765:8080
    environment:
      JAVA_OPTS: "-Drocketmq.namesrv.addr=rmqnamesrv:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false"
    depends_on:
      - rmqnamesrv
    networks:
      rmq:
        aliases:
          - rmqconsole

networks:
  rmq:
    name: rmq
    driver: bridge

当我用你的默认配置不改动直接执行sh脚本的时候,opt/log目录下时会有日志文件产生的,但是我只改掉宿主机映射路径的时候,我进入容器的opt/log下可以找到路径,单却没有任何日志文件产生,真的把我整懵了

IDEA部署完的路径映射
image

部署时,能不能不用777初始化目录?

# start.sh文件中:

# 设置目录权限
chmod -R 777 ./rmqs/logs
chmod -R 777 ./rmqs/store
chmod -R 777 ./rmq/logs
chmod -R 777 ./rmq/store

能不能用更小的权限来初始化目录?

用了777,平台的运维同事直接驳回了。
有安全风险,主机上任何用户都可以改写目录。

在web页面点击消费者菜单栏,后台会报错

web界面:
微信图片_20200203184935

错误信息如下:

[2020-02-03 17:18:02.598]  WARN examineConsumerConnectionInfo exception, FILTERSRV_CONSUMER
org.apache.rocketmq.client.exception.MQClientException: CODE: 17  DESC: No topic route info in name server for the topic: %RETRY%FILTERSRV_CONSUMER
See http://rocketmq.apache.org/docs/faq/ for further details.
    at org.apache.rocketmq.client.impl.MQClientAPIImpl.getTopicRouteInfoFromNameServer(MQClientAPIImpl.java:1212)
    at org.apache.rocketmq.tools.admin.DefaultMQAdminExtImpl.examineTopicRouteInfo(DefaultMQAdminExtImpl.java:274)
    at org.apache.rocketmq.tools.admin.DefaultMQAdminExtImpl.examineConsumerConnectionInfo(DefaultMQAdminExtImpl.java:293)
    at org.apache.rocketmq.tools.admin.DefaultMQAdminExt.examineConsumerConnectionInfo(DefaultMQAdminExt.java:223)
    at org.apache.rocketmq.console.service.client.MQAdminExtImpl.examineConsumerConnectionInfo(MQAdminExtImpl.java:188)

docker-compose.yml 配置:

version: '3.5'
services:
  rmqnamesrv:
    image: foxiswho/rocketmq:server
    container_name: rmqnamesrv
    ports:
      - 9876:9876
    volumes:
      - ./data/logs:/opt/logs
      - ./data/store:/opt/store
    networks:
        rmq:
          aliases:
            - rmqnamesrv

  rmqbroker:
    image: foxiswho/rocketmq:broker
    container_name: rmqbroker
    ports:
      - 10909:10909
      - 10911:10911
    volumes:
      - ./data/logs:/opt/logs
      - ./data/store:/opt/store
      - ./data/brokerconf/broker.conf:/etc/rocketmq/broker.conf
    environment:
        NAMESRV_ADDR: "rmqnamesrv:9876"
        JAVA_OPTS: " -Duser.home=/opt"
        JAVA_OPT_EXT: "-server -Xms128m -Xmx128m -Xmn128m"
    command: mqbroker -c /etc/rocketmq/broker.conf
    depends_on:
      - rmqnamesrv
    networks:
      rmq:
        aliases:
          - rmqbroker

  rmqconsole:
    image: styletang/rocketmq-console-ng
    container_name: rmqconsole
    ports:
      - 8080:8080
    environment:
        JAVA_OPTS: "-Drocketmq.namesrv.addr=rmqnamesrv:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false"
    depends_on:
      - rmqnamesrv
    networks:
      rmq:
        aliases:
          - rmqconsole

networks:
  rmq:
    name: rmq
    driver: bridge

broker.conf配置:

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.


# 所属集群名字
brokerClusterName=DefaultCluster

# broker 名字,注意此处不同的配置文件填写的不一样,如果在 broker-a.properties 使用: broker-a,
# 在 broker-b.properties 使用: broker-b
brokerName=broker-a

# 0 表示 Master,> 0 表示 Slave
brokerId=0

# nameServer地址,分号分割
#namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
namesrvAddr=192.168.1.12:9876


# 启动IP,如果 docker 报 com.alibaba.rocketmq.remoting.exception.RemotingConnectException: connect to <x.x.x.x:10909> failed
# 解决方式1 加上一句 producer.setVipChannelEnabled(false);,解决方式2 brokerIP1 设置宿主机IP,不要使用docker 内部IP
brokerIP1=192.168.1.12

# 在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4

# 是否允许 Broker 自动创建 Topic,建议线下开启,线上关闭 !!!
autoCreateTopicEnable=true

# 是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true

# Broker 对外服务的监听端口
listenPort=10911

# 删除文件时间点,默认凌晨4点
deleteWhen=04

# 文件保留时间,默认48小时
fileReservedTime=120

# commitLog 每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824

# ConsumeQueue 每个文件默认存 30W 条,根据业务情况调整
mapedFileSizeConsumeQueue=300000

# destroyMapedFileIntervalForcibly=120000
# redeleteHangedFileInterval=120000
# 检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
# 存储路径
# storePathRootDir=/usr/local/rocketMQ/datas
# commitLog 存储路径
# storePathCommitLog=/usr/local/rocketMQ/datas/commitlog
# 消费队列存储
# storePathConsumeQueue=/usr/local/rocketMQ/datas/consumequeue
# 消息索引存储路径
# storePathIndex=/usr/local/rocketMQ/datas/index
# checkpoint 文件存储路径
# storeCheckpoint=/usr/local/rocketMQ/datas/checkpoint
# abort 文件存储路径
# abortFile=/usr/local/rocketMQ/datas/abort
# 限制的消息大小
maxMessageSize=65536

# flushCommitLogLeastPages=4
# flushConsumeQueueLeastPages=2
# flushCommitLogThoroughInterval=10000
# flushConsumeQueueThoroughInterval=60000

# Broker 的角色
# - ASYNC_MASTER 异步复制Master
# - SYNC_MASTER 同步双写Master
# - SLAVE
brokerRole=ASYNC_MASTER

# 刷盘方式
# - ASYNC_FLUSH 异步刷盘
# - SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH

# 发消息线程池数量
# sendMessageThreadPoolNums=128
# 拉消息线程池数量
# pullMessageThreadPoolNums=128

4.8版本和4.7版本配置内存无效

你好,在4.8版本和4.7版本配置内存无效,
如:在docker-compose中配置了JAVA_OPT_EXT=-Xms128m -Xmx128m -Xmn128m参数后,容器启动时未使用我刚指定的配置,希望能在dockerfile中加一下ENV环境变量
配置:
image
容器中的内存:
image

java.io.FileNotFoundException: /etc/rocketmq/broker.conf (Is a directory)

-server -Xms491M -Xmx491M -Xmn122M -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=491M -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-0.el7_9.x86_64/jre/lib/ext:/home/rocketmq/rocketmq-4.8.0/bin/../lib -Duser.home=/home/rocketmq -Xms512M -Xmx512M -Xmn128m -cp .:/home/rocketmq/rocketmq-4.8.0/bin/../conf:

java.io.FileNotFoundException: /etc/rocketmq/broker.conf (Is a directory)

at java.io.FileInputStream.open0(Native Method)

at java.io.FileInputStream.open(FileInputStream.java:195)

at java.io.FileInputStream.(FileInputStream.java:138)

at java.io.FileInputStream.(FileInputStream.java:93)

at org.apache.rocketmq.broker.BrokerStartup.createBrokerController(BrokerStartup.java:128)

at org.apache.rocketmq.broker.BrokerStartup.main(BrokerStartup.java:58)

-server -Xms994M -Xmx994M -Xmn200M -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=994M -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.275.b01-0.el7_9.x86_64/jre/lib/ext:/home/rocketmq/rocketmq-4.8.0/bin/../lib -Duser.home=/home/rocketmq -Xms512M -Xmx512M -Xmn128m -cp .:/home/rocketmq/rocketmq-4.8.0/bin/../conf:

java.io.FileNotFoundException: /home/rocketmq/conf/broker.conf

docker 命令里文件地址配错了
docker run -d -v $(pwd)/logs:/home/rocketmq/logs -v $(pwd)/store:/home/rocketmq/store \

  -v $(pwd)/conf:/home/rocketmq/conf \
  --name rmqbroker \
  -e "NAMESRV_ADDR=rmqnamesrv:9876" \
  -e "JAVA_OPT_EXT=-Xms512M -Xmx512M -Xmn128m" \
  -p 10911:10911 -p 10912:10912 -p 10909:10909 \
  foxiswho/rocketmq:4.8.0 \
  sh mqbroker -c /home/rocketmq/conf/broker.conf

注意几个地方,要不会启动失败

  1. broker 目录映射,不仅logs目录要设置777权限,conf和store也要设置
  2. console部署时,环境变量JAVA_OPTS=-Drocketmq.namesrv.addr=rmqserver:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false要改为 JAVA_OPTS=-Drocketmq.namesrv.addr=rmqnamesrv:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false

rmq镜像里的runBroker.sh可能有误

Dynamically calculate parameters, for reference.

Xms=$MAX_HEAP_SIZE ($MIN_HEAP_SIZE?)
Xmx=$MAX_HEAP_SIZE
Xmn=$HEAP_NEWSIZE
MaxDirectMemorySize=$MAX_HEAP_SIZE

rocket-broker启动后容器关闭Exist 253

我使用环境变量
JAVA_OPT_EXT: "-Duser.home=/opt -server -Xms512M -Xmx512M -Xmn128m" 时,broker无法启动,控制台打印:

broker      | ======================================
broker      | -server -Xms3971M -Xmx3971M -Xmn800M -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=3971M -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/ext:/home/rocketmq/rocketmq-4.5.2/bin/../lib -cp .:/home/rocketmq/rocketmq-4.5.2/bin/../conf:

然后我把环境变量改为:
JAVA_OPTS: "-Duser.home=/opt -server -Xms512M -Xmx512M -Xmn128m"
后启动成功。控制台还是打印

broker      | ======================================
broker      | -server -Xms3971M -Xmx3971M -Xmn800M -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=3971M -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/ext:/home/rocketmq/rocketmq-4.5.2/bin/../lib -cp .:/home/rocketmq/rocketmq-4.5.2/bin/../conf:

这是什么原因,有什么影响吗?我看到Xmx还是4个G

broker exited with code 253

按 rmq/docker-compose.yml 下的配置去开启 broker 一直启动不起来,exited with code 253,然后把 docker-compose.yml 中配置的

JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"

改为

JAVA_OPTS: " -Duser.home=/opt" 
JAVA_OPT_EXT: "-server -Xms512M -Xmx512M -Xmn128m"

就能正常启动了

不能启动的完整 docker-compose.yml

version: '3.5'
services:
  rmqnamesrv:
    image: foxiswho/rocketmq:4.7.0
    container_name: rmqnamesrv
    ports:
      - 9876:9876
    environment:
        TZ: Asia/Shanghai
        JAVA_OPT_EXT: "-server -Xms512m -Xmx512m -Xmn128m"
    volumes:
      - /tmp/rocketmq/logs/nameserver:/home/rocketmq/logs
    command: sh mqnamesrv
    networks:
        rmq:
          aliases:
            - rmqnamesrv

  rmqbroker-a:
    image: foxiswho/rocketmq:4.7.0
    container_name: rmqbroker-a
    ports:
      - 10911:10911
      # - 10909:10909
    volumes:
      - /tmp/rocketmq/logs/broker-a:/home/rocketmq/logs
      - /tmp/rocketmq/store/broker-a:/home/rocketmq/store
      - /tmp/broker-a.conf:/etc/rocketmq/broker.conf
    environment:
        TZ: Asia/Shanghai
        JAVA_OPT_EXT: "-Duser.home=/opt -Xms512M -Xmx512M -Xmn128m"
    command: ["sh","mqbroker","-c","/etc/rocketmq/broker.conf","-n","rmqnamesrv:9876","autoCreateTopicEnable=true"]
    depends_on:
      - rmqnamesrv
    networks:
      rmq:
        aliases:
          - rmqbroker-a
          
networks:
  rmq:
    name: rmq
    driver: bridge

环境信息

# 镜像版本
foxiswho/rocketmq:4.7.0

# 操作系统
Darwin Jmac.local 19.4.0 Darwin Kernel Version 19.4.0: Wed Mar  4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64 x86_64`

# docker 版本
Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b
 Built:             Wed Mar 11 01:21:11 2020
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       afacb8b
  Built:            Wed Mar 11 01:29:16 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

请问下如何能把消息目录挂载到本地磁盘上,按照传统挂载方式都启动不了,现在一重启docker数据就没了

broker.conf 如下

存储路径

storePathRootDir=/opt/rkmq/store

commitLog 存储路径

storePathCommitLog=/opt/rkmq/commitlog

消费队列存储

storePathConsumeQueue=/opt/rkmq/consumequeue

消息索引存储路径

storePathIndex=/opt/rkmq/index

checkpoint 文件存储路径

storeCheckpoint=/opt/rkmq/checkpoint

abort 文件存储路径

abortFile=/opt/rkmq/abort

dockercompose 如果加上下面的内容挂载到本地磁盘上就启动不来 报rmqbroker exited with code 253
- ./data/rkmq/store:/opt/rkmq/store
# # commitLog 存储路径
- ./data/rkmq/commitLog:/opt/rkmq/commitlog
# # 消费队列存储
- ./data/rkmq/consumequeue:/opt/rkmq/consumequeue
# # 消息索引存储路径
- ./data/rkmq/index:/opt/rkmq/index
# # checkpoint 文件存储路径
- ./data/rkmq/checkpoint:/opt/rkmq/checkpoint
# # abort 文件存储路径
- ./data/rkmq/abort:/opt/rkmq/abort

镜像是可以运行的 但是出现了RemotingTooMuchRequestException这个异常

RocketMQLog:WARN No appenders could be found for logger (io.netty.util.internal.InternalThreadLocalMap).
RocketMQLog:WARN Please initialize the logger system properly.
Exception in thread "main" org.apache.rocketmq.remoting.exception.RemotingTooMuchRequestException: sendDefaultImpl call timeout
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:640)
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1310)
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1256)
at org.apache.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:339)
at Test.main(Test.java:28)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.