Comments (25)
coffee-squirrel
commented on June 12, 2024
2
Since you're on Docker 20.10.5 (released 2021-03-02) and 20.10.8 (released 2021-08-03), I'd suggest trying to get that upgraded to at least 20.10.10 (released 2021-10-25; latest is 20.10.17 released 2022-06-06). Doing a bit of searching on the second issue, I found https://stackoverflow.com/a/72841934 (and therefore adoptium/containers#215), which seems like it could be related (potentially to both issues) given 4.3.4 is now based upon the Jammy/22.0.4 variant of the eclipse-temurin images.
from graylog-docker.
pschichtel
commented on June 12, 2024
1
@mpfz0r I think the profile-script thing is a red herring. The root cause is the fact that the new eclipse-temurin base-image uses a new glibc version that uses the clone3 syscall, which is blocked by docker's default seccomp policy, which was updated with 20.10.10. so all older docker versions will fail with the same issue.
If you are able to downgrade your docker version to 20.10.9 or older you should be able to reproduce this issue.
from graylog-docker.
mpfz0r
commented on June 12, 2024
1
Can you update your docker version?
If not, you can try running it unconfined as workaround:
https://stackoverflow.com/questions/46053672/set-secomp-to-unconfined-in-docker-compose
from graylog-docker.
mpfz0r
commented on June 12, 2024
@MahdiGhiasi
could you run your graylog in docker-compose with a debugging entry point for us and give us the output?
entrypoint: "/bin/bash -c 'find /usr/share/graylog -type d -ls'"
from graylog-docker.
mpfz0r
commented on June 12, 2024
This seems to be where this happens:
But I don't see why there is a regression between 4.3.3 and 4.3.4
https://github.com/Graylog2/graylog-docker/blob/4.3/docker-entrypoint.sh#L91-L94
# Create data directories
for d in journal log plugin config contentpacks
do
dir=${GRAYLOG_HOME}/data/${d}
[[ -d "${dir}" ]] || mkdir -p "${dir}"
if [[ "$(stat --format='%U:%G' $dir)" != 'graylog:graylog' ]] && [[ -w "$dir" ]]; then
chown -R graylog:graylog "$dir" || echo "Warning can not change owner to graylog:graylog"
fi
done
}
from graylog-docker.
coffee-squirrel
commented on June 12, 2024
We're currently running 2 environments on the 4.3.4 images (1 OSS, 1 Enterprise) and haven't ran into this yet.
mkdir: cannot create directory ‘/data’ almost makes it seem like ${GRAYLOG_HOME} isn't set. So far I've only seen that type of message with something like mkdir -p /data/foo (or /data, of course).
from graylog-docker.
MahdiGhiasi
commented on June 12, 2024
@mpfz0r Here's the output for the debugging entry point you requested:
graylog_1 | 1317781 4 drwxr-xr-x 8 graylog graylog 4096 Aug 3 19:02 /usr/share/graylog
graylog_1 | 1317784 4 drwxr-xr-x 2 graylog graylog 4096 Aug 3 19:02 /usr/share/graylog/bin
graylog_1 | 1317797 4 drwxr-xr-x 2 graylog graylog 4096 Aug 3 19:02 /usr/share/graylog/plugin
graylog_1 | 1317805 4 drwxr-xr-x 2 graylog graylog 4096 Aug 3 19:02 /usr/share/graylog/plugins-merged
graylog_1 | 527453 4 drwxr-x--- 7 graylog graylog 4096 Aug 4 17:26 /usr/share/graylog/data
graylog_1 | 527460 4 drwxr-x--- 2 graylog graylog 4096 Aug 3 19:02 /usr/share/graylog/data/plugin
graylog_1 | 1703937 4 drwxrwxrwx 3 graylog graylog 4096 Aug 4 04:20 /usr/share/graylog/data/journal
graylog_1 | 1703938 4 drwxr-xr-x 2 graylog graylog 4096 Aug 4 04:01 /usr/share/graylog/data/journal/messagejournal-0
graylog_1 | 527457 4 drwxr-x--- 2 graylog graylog 4096 Aug 3 19:02 /usr/share/graylog/data/data
graylog_1 | 527459 4 drwxr-x--- 2 graylog graylog 4096 Aug 3 19:02 /usr/share/graylog/data/log
graylog_1 | 527454 4 drwxr-x--- 2 graylog graylog 4096 Aug 4 17:26 /usr/share/graylog/data/config
graylog_1 | 1317796 4 drwxr-xr-x 2 graylog graylog 4096 Aug 3 18:01 /usr/share/graylog/log
graylog_1 | 1317798 4 drwxr-xr-x 2 graylog graylog 4096 Aug 3 19:02 /usr/share/graylog/plugins-default
from graylog-docker.
MahdiGhiasi
commented on June 12, 2024
I've also confirmed again that docker image graylog/graylog:4.3.3 can start properly but graylog/graylog:4.3.4 gives the error I described, in our environment (nothing is changed in our environment other than the docker image tag).
I've rolled back our production Graylog instance to 4.3.3 for now and it runs properly; but I'm happy to do any tests or give any details that might help finding the root cause of this in 4.3.4 on our environment.
from graylog-docker.
mpfz0r
commented on June 12, 2024
@MahdiGhiasi Thanks.
Could you run entrypoint: "/bin/bash -xv /docker-entrypoint.sh"
for me?
I think @coffee-squirrel might be right. the /data error output is odd
from graylog-docker.
MahdiGhiasi
commented on June 12, 2024
@mpfz0r There you go:
graylog_1 | + grep -q UseConcMarkSweepGC
graylog_1 | + /opt/java/openjdk/bin/java -XX:+PrintFlagsFinal
graylog_1 | adding environment opts
graylog_1 |
graylog_1 | # and add the previous saved settings to our defaults
graylog_1 | if [[ ! -z ${__GRAYLOG_SERVER_JAVA_OPTS} ]]
graylog_1 | then
graylog_1 | echo "adding environment opts"
graylog_1 | GRAYLOG_SERVER_JAVA_OPTS="${GRAYLOG_SERVER_JAVA_OPTS} ${__GRAYLOG_SERVER_JAVA_OPTS}"
graylog_1 | export GRAYLOG_SERVER_JAVA_OPTS
graylog_1 | fi
graylog_1 | + [[ ! -z -Xms256m -Xmx256m ]]
graylog_1 | + echo 'adding environment opts'
graylog_1 | + GRAYLOG_SERVER_JAVA_OPTS='-Xms256m -Xmx256m -Xms256m -Xmx256m'
graylog_1 | + export GRAYLOG_SERVER_JAVA_OPTS
graylog_1 |
graylog_1 | # Convert all environment variables with names ending in __FILE into the content of
graylog_1 | # the file that they point at and use the name without the trailing __FILE.
graylog_1 | # This can be used to carry in Docker secrets.
graylog_1 | for VAR_NAME in $(env | grep '^GRAYLOG_[^=]\+__FILE=.\+' | sed -r 's/^(GRAYLOG_[^=]*)__FILE=.*/\1/g'); do
graylog_1 | VAR_NAME_FILE="${VAR_NAME}__FILE"
graylog_1 | if [ "${!VAR_NAME}" ]; then
graylog_1 | echo >&2 "ERROR: Both ${VAR_NAME} and ${VAR_NAME_FILE} are set but are exclusive"
graylog_1 | exit 1
graylog_1 | fi
graylog_1 | VAR_FILENAME="${!VAR_NAME_FILE}"
graylog_1 | echo "Getting secret ${VAR_NAME} from ${VAR_FILENAME}"
graylog_1 | if [ ! -r "${VAR_FILENAME}" ]; then
graylog_1 | echo >&2 "ERROR: ${VAR_FILENAME} does not exist or is not readable"
graylog_1 | exit 1
graylog_1 | fi
graylog_1 | export "${VAR_NAME}"="$(< "${VAR_FILENAME}")"
graylog_1 | unset "${VAR_NAME_FILE}"
graylog_1 | done
graylog_1 | ++ env
graylog_1 | ++ sed -r 's/^(GRAYLOG_[^=]*)__FILE=.*/\1/g'
graylog_1 | ++ grep '^GRAYLOG_[^=]\+__FILE=.\+'
graylog_1 |
graylog_1 |
graylog_1 | # Delete outdated PID file
graylog_1 | [[ -e /tmp/graylog.pid ]] && rm --force /tmp/graylog.pid
graylog_1 | + [[ -e /tmp/graylog.pid ]]
graylog_1 |
graylog_1 | # check if we are inside kubernetes, Graylog should be run as statefulset and $POD_NAME env var should be defined like this
graylog_1 | # env:
graylog_1 | # - name: POD_NAME
graylog_1 | # valueFrom:
graylog_1 | # fieldRef:
graylog_1 | # fieldPath: metadata.name
graylog_1 | # First stateful member is having pod name ended with -0, so
graylog_1 | if [[ ! -z "${POD_NAME}" ]]
graylog_1 | then
graylog_1 | if echo "${POD_NAME}" | grep "\\-0$" >/dev/null
graylog_1 | then
graylog_1 | export GRAYLOG_IS_MASTER="true"
graylog_1 | else
graylog_1 | export GRAYLOG_IS_MASTER="false"
graylog_1 | fi
graylog_1 | fi
graylog_1 | + [[ ! -z '' ]]
graylog_1 |
graylog_1 | # check if we are inside a nomad cluster
graylog_1 | # First member is having alloc-index 0, so
graylog_1 | if [[ ! -z "${NOMAD_ALLOC_INDEX}" ]]; then
graylog_1 | if [ ${NOMAD_ALLOC_INDEX} == 0 ]; then
graylog_1 | export GRAYLOG_IS_MASTER="true"
graylog_1 | else
graylog_1 | export GRAYLOG_IS_MASTER="false"
graylog_1 | fi
graylog_1 | fi
graylog_1 | + [[ ! -z '' ]]
graylog_1 |
graylog_1 | # Merge plugin dirs to allow mounting of /plugin as a volume
graylog_1 | export GRAYLOG_PLUGIN_DIR=/usr/share/graylog/plugins-merged
graylog_1 | + export GRAYLOG_PLUGIN_DIR=/usr/share/graylog/plugins-merged
graylog_1 | + GRAYLOG_PLUGIN_DIR=/usr/share/graylog/plugins-merged
graylog_1 | rm -f /usr/share/graylog/plugins-merged/*
graylog_1 | + rm -f /usr/share/graylog/plugins-merged/graylog-plugin-aws-4.3.4.jar /usr/share/graylog/plugins-merged/graylog-plugin-collector-4.3.4.jar /usr/share/graylog/plugins-merged/graylog-plugin-integrations-4.3.4.jar /usr/share/graylog/plugins-merged/graylog-plugin-threatintel-4.3.4.jar /usr/share/graylog/plugins-merged/graylog-storage-elasticsearch6-4.3.4.jar /usr/share/graylog/plugins-merged/graylog-storage-elasticsearch7-4.3.4.jar
graylog_1 | find /usr/share/graylog/plugins-default/ -type f -exec cp {} /usr/share/graylog/plugins-merged/ \;
graylog_1 | + find /usr/share/graylog/plugins-default/ -type f -exec cp '{}' /usr/share/graylog/plugins-merged/ ';'
graylog_1 | find /usr/share/graylog/plugin/ -type f -exec cp {} /usr/share/graylog/plugins-merged/ \;
graylog_1 | + find /usr/share/graylog/plugin/ -type f -exec cp '{}' /usr/share/graylog/plugins-merged/ ';'
graylog_1 |
graylog_1 |
graylog_1 | setup() {
graylog_1 | # Create data directories
graylog_1 | for d in journal log plugin config contentpacks
graylog_1 | do
graylog_1 | dir=${GRAYLOG_HOME}/data/${d}
graylog_1 | [[ -d "${dir}" ]] || mkdir -p "${dir}"
graylog_1 |
graylog_1 | if [[ "$(stat --format='%U:%G' $dir)" != 'graylog:graylog' ]] && [[ -w "$dir" ]]; then
graylog_1 | chown -R graylog:graylog "$dir" || echo "Warning can not change owner to graylog:graylog"
graylog_1 | fi
graylog_1 | done
graylog_1 | }
graylog_1 |
graylog_1 | graylog() {
graylog_1 |
graylog_1 | exec "${JAVA_HOME}/bin/java" \
graylog_1 | ${GRAYLOG_SERVER_JAVA_OPTS} \
graylog_1 | -jar \
graylog_1 | -Dlog4j.configurationFile="${GRAYLOG_HOME}/data/config/log4j2.xml" \
graylog_1 | -Djava.library.path="${GRAYLOG_HOME}/lib/sigar/" \
graylog_1 | -Dgraylog2.installation_source=docker \
graylog_1 | "${GRAYLOG_HOME}/graylog.jar" \
graylog_1 | "$@" \
graylog_1 | -f "${GRAYLOG_HOME}/data/config/graylog.conf"
graylog_1 | }
graylog_1 |
graylog_1 | run() {
graylog_1 | setup
graylog_1 |
graylog_1 | # if being called without an argument assume "server" for backwards compatibility
graylog_1 | if [ $# = 0 ]; then
graylog_1 | graylog server "$@"
graylog_1 | fi
graylog_1 |
graylog_1 | graylog "$@"
graylog_1 | }
graylog_1 |
graylog_1 | run "$@"
graylog_1 | + run
graylog_1 | + setup
graylog_1 | + for d in journal log plugin config contentpacks
graylog_1 | + dir=/data/journal
graylog_1 | + [[ -d /data/journal ]]
graylog_1 | + mkdir -p /data/journal
graylog_1 | mkdir: cannot create directory ‘/data’: Permission denied
graylog_graylog_1 exited with code 1
It seems that @coffee-squirrel is right, the last few lines seem to indicate that ${GRAYLOG_HOME} is not being set properly.
from graylog-docker.
mpfz0r
commented on June 12, 2024
@MahdiGhiasi Yeah, looks like it. But your output is truncated. It misses the part where /etc/profile is sourced, that's where GRAYLOG_HOME should be set
from graylog-docker.
MahdiGhiasi
commented on June 12, 2024
@mpfz0r Oh, sorry. Here's the complete output:
#!/bin/bash
set -e
+ set -e
# save the settings over the docker(-compose) environment
__GRAYLOG_SERVER_JAVA_OPTS=${GRAYLOG_SERVER_JAVA_OPTS}
+ __GRAYLOG_SERVER_JAVA_OPTS='-Xms256m -Xmx256m'
# shellcheck disable=SC1091
source /etc/profile
+ source /etc/profile
# /etc/profile: system-wide .profile file for the Bourne shell (sh(1))
# and Bourne compatible shells (bash(1), ksh(1), ash(1), ...).
if [ "${PS1-}" ]; then
if [ "${BASH-}" ] && [ "$BASH" != "/bin/sh" ]; then
# The file bash.bashrc already sets the default PS1.
# PS1='\h:\w\$ '
if [ -f /etc/bash.bashrc ]; then
. /etc/bash.bashrc
fi
else
if [ "$(id -u)" -eq 0 ]; then
PS1='# '
else
PS1='$ '
fi
fi
fi
++ '[' '' ']'
if [ -d /etc/profile.d ]; then
for i in /etc/profile.d/*.sh; do
if [ -r $i ]; then
. $i
fi
done
unset i
fi
++ '[' -d /etc/profile.d ']'
++ for i in /etc/profile.d/*.sh
++ '[' -r /etc/profile.d/01-locale-fix.sh ']'
++ for i in /etc/profile.d/*.sh
++ '[' -r /etc/profile.d/graylog.sh ']'
++ unset i
#Set default GC
if [[ -z ${GRAYLOG_DOCKER_DISABLE_CMS_GC} ]]
then
if "${JAVA_HOME}/bin/java" -XX:+PrintFlagsFinal 2>&1 |grep -q UseParNewGC; then
GRAYLOG_SERVER_JAVA_OPTS="${GRAYLOG_SERVER_JAVA_OPTS} -XX:+UseParNewGC"
export GRAYLOG_SERVER_JAVA_OPTS
fi
if "${JAVA_HOME}/bin/java" -XX:+PrintFlagsFinal 2>&1 |grep -q UseConcMarkSweepGC; then
GRAYLOG_SERVER_JAVA_OPTS="${GRAYLOG_SERVER_JAVA_OPTS} -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled"
export GRAYLOG_SERVER_JAVA_OPTS
fi
fi
+ [[ -z '' ]]
+ /opt/java/openjdk/bin/java -XX:+PrintFlagsFinal
+ grep -q UseParNewGC
+ /opt/java/openjdk/bin/java -XX:+PrintFlagsFinal
+ grep -q UseConcMarkSweepGC
# and add the previous saved settings to our defaults
if [[ ! -z ${__GRAYLOG_SERVER_JAVA_OPTS} ]]
then
echo "adding environment opts"
GRAYLOG_SERVER_JAVA_OPTS="${GRAYLOG_SERVER_JAVA_OPTS} ${__GRAYLOG_SERVER_JAVA_OPTS}"
export GRAYLOG_SERVER_JAVA_OPTS
fi
+ [[ ! -z -Xms256m -Xmx256m ]]
+ echo 'adding environment opts'
adding environment opts
+ GRAYLOG_SERVER_JAVA_OPTS='-Xms256m -Xmx256m -Xms256m -Xmx256m'
+ export GRAYLOG_SERVER_JAVA_OPTS
# Convert all environment variables with names ending in __FILE into the content of
# the file that they point at and use the name without the trailing __FILE.
# This can be used to carry in Docker secrets.
for VAR_NAME in $(env | grep '^GRAYLOG_[^=]\+__FILE=.\+' | sed -r 's/^(GRAYLOG_[^=]*)__FILE=.*/\1/g'); do
VAR_NAME_FILE="${VAR_NAME}__FILE"
if [ "${!VAR_NAME}" ]; then
echo >&2 "ERROR: Both ${VAR_NAME} and ${VAR_NAME_FILE} are set but are exclusive"
exit 1
fi
VAR_FILENAME="${!VAR_NAME_FILE}"
echo "Getting secret ${VAR_NAME} from ${VAR_FILENAME}"
if [ ! -r "${VAR_FILENAME}" ]; then
echo >&2 "ERROR: ${VAR_FILENAME} does not exist or is not readable"
exit 1
fi
export "${VAR_NAME}"="$(< "${VAR_FILENAME}")"
unset "${VAR_NAME_FILE}"
done
++ sed -r 's/^(GRAYLOG_[^=]*)__FILE=.*/\1/g'
++ grep '^GRAYLOG_[^=]\+__FILE=.\+'
++ env
# Delete outdated PID file
[[ -e /tmp/graylog.pid ]] && rm --force /tmp/graylog.pid
+ [[ -e /tmp/graylog.pid ]]
# check if we are inside kubernetes, Graylog should be run as statefulset and $POD_NAME env var should be defined like this
# env:
# - name: POD_NAME
# valueFrom:
# fieldRef:
# fieldPath: metadata.name
# First stateful member is having pod name ended with -0, so
if [[ ! -z "${POD_NAME}" ]]
then
if echo "${POD_NAME}" | grep "\\-0$" >/dev/null
then
export GRAYLOG_IS_MASTER="true"
else
export GRAYLOG_IS_MASTER="false"
fi
fi
+ [[ ! -z '' ]]
# check if we are inside a nomad cluster
# First member is having alloc-index 0, so
if [[ ! -z "${NOMAD_ALLOC_INDEX}" ]]; then
if [ ${NOMAD_ALLOC_INDEX} == 0 ]; then
export GRAYLOG_IS_MASTER="true"
else
export GRAYLOG_IS_MASTER="false"
fi
fi
+ [[ ! -z '' ]]
# Merge plugin dirs to allow mounting of /plugin as a volume
export GRAYLOG_PLUGIN_DIR=/usr/share/graylog/plugins-merged
+ export GRAYLOG_PLUGIN_DIR=/usr/share/graylog/plugins-merged
+ GRAYLOG_PLUGIN_DIR=/usr/share/graylog/plugins-merged
rm -f /usr/share/graylog/plugins-merged/*
+ rm -f '/usr/share/graylog/plugins-merged/*'
find /usr/share/graylog/plugins-default/ -type f -exec cp {} /usr/share/graylog/plugins-merged/ \;
+ find /usr/share/graylog/plugins-default/ -type f -exec cp '{}' /usr/share/graylog/plugins-merged/ ';'
find /usr/share/graylog/plugin/ -type f -exec cp {} /usr/share/graylog/plugins-merged/ \;
+ find /usr/share/graylog/plugin/ -type f -exec cp '{}' /usr/share/graylog/plugins-merged/ ';'
setup() {
# Create data directories
for d in journal log plugin config contentpacks
do
dir=${GRAYLOG_HOME}/data/${d}
[[ -d "${dir}" ]] || mkdir -p "${dir}"
if [[ "$(stat --format='%U:%G' $dir)" != 'graylog:graylog' ]] && [[ -w "$dir" ]]; then
chown -R graylog:graylog "$dir" || echo "Warning can not change owner to graylog:graylog"
fi
done
}
graylog() {
exec "${JAVA_HOME}/bin/java" \
${GRAYLOG_SERVER_JAVA_OPTS} \
-jar \
-Dlog4j.configurationFile="${GRAYLOG_HOME}/data/config/log4j2.xml" \
-Djava.library.path="${GRAYLOG_HOME}/lib/sigar/" \
-Dgraylog2.installation_source=docker \
"${GRAYLOG_HOME}/graylog.jar" \
"$@" \
-f "${GRAYLOG_HOME}/data/config/graylog.conf"
}
run() {
setup
# if being called without an argument assume "server" for backwards compatibility
if [ $# = 0 ]; then
graylog server "$@"
fi
graylog "$@"
}
run "$@"
+ run
+ setup
+ for d in journal log plugin config contentpacks
+ dir=/data/journal
+ [[ -d /data/journal ]]
+ mkdir -p /data/journal
mkdir: cannot create directory ‘/data’: Permission denied
from graylog-docker.
mpfz0r
commented on June 12, 2024
@MahdiGhiasi
looks like /etc/profile.d/graylog.sh is not readable for some reason.
what is the output of entrypoint: "/bin/bash -c 'ls -l /etc/profile.d/'"
from graylog-docker.
MahdiGhiasi
commented on June 12, 2024
graylog_1 | total 8
graylog_1 | -rw-r--r-- 1 root root 96 Oct 15 2021 01-locale-fix.sh
graylog_1 | -rw-r--r-- 1 root root 564 Aug 3 19:02 graylog.sh
from graylog-docker.
mpfz0r
commented on June 12, 2024
@MahdiGhiasi entrypoint: "/bin/bash -c 'ls -ld /etc/profile.d/; ls -ld /etc; ls -ld /'"
from graylog-docker.
MahdiGhiasi
commented on June 12, 2024
graylog_1 | drwxr-xr-x 1 root root 4096 Aug 3 19:02 /etc/profile.d/
graylog_1 | drwxr-xr-x 1 root root 4096 Aug 4 18:24 /etc
graylog_1 | drwxr-xr-x 1 root root 4096 Aug 4 18:24 /
from graylog-docker.
mpfz0r
commented on June 12, 2024
@MahdiGhiasi entrypoint: "/bin/bash -c '[ -r /etc/profile.d/graylog.sh ] && echo YEAH'"
from graylog-docker.
MahdiGhiasi
commented on June 12, 2024
@mpfz0r This does not print YEAH.
from graylog-docker.
mpfz0r
commented on June 12, 2024
@MahdiGhiasi hmm, that's weird.. And for now I'm out of ideas. Maybe some one else?
Which docker version are you running?
from graylog-docker.
MahdiGhiasi
commented on June 12, 2024
@mpfz0r Docker version 20.10.5 (build 55c4c88), running on Ubuntu 20.04.1 LTS.
On a maybe related note, I've also tried to upgrade another machine to Graylog 4.3.4 from 4.3.3, this one also fails but for an entirely different reason! (This machine is running Docker version 20.10.8 build 3967b7d, Ubuntu 20.04.2 LTS)
This one passes the adding environment opts successfully, but java fails to start in the container due to insufficient memory.
However, heap size is set to 1.5GB (-Xms1536m -Xmx1536m) and there's at least 8GB free memory on this server, so I don't know why it's complaining about that.
And this one also works fine on 4.3.3, but breaks on 4.3.4.
Here's the log for that:
graylog_1 | [0.003s][warning][os,thread] Failed to start thread "GC Thread#0" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.
graylog_1 | #
graylog_1 | # There is insufficient memory for the Java Runtime Environment to continue.
graylog_1 | # Cannot create worker GC thread. Out of system resources.
graylog_1 | # An error report file with more information is saved as:
graylog_1 | # /usr/share/graylog/hs_err_pid8.log
graylog_1 | adding environment opts
graylog_1 | [0.003s][warning][os,thread] Failed to start thread "GC Thread#0" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.
graylog_1 | #
graylog_1 | # There is insufficient memory for the Java Runtime Environment to continue.
graylog_1 | # Cannot create worker GC thread. Out of system resources.
graylog_1 | # Can not save log file, dump to screen..
graylog_1 | #
graylog_1 | # There is insufficient memory for the Java Runtime Environment to continue.
graylog_1 | # Cannot create worker GC thread. Out of system resources.
graylog_1 | # Possible reasons:
graylog_1 | # The system is out of physical RAM or swap space
graylog_1 | # The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
graylog_1 | # Possible solutions:
graylog_1 | # Reduce memory load on the system
graylog_1 | # Increase physical memory or swap space
graylog_1 | # Check if swap backing store is full
graylog_1 | # Decrease Java heap size (-Xmx/-Xms)
graylog_1 | # Decrease number of Java threads
graylog_1 | # Decrease Java thread stack sizes (-Xss)
graylog_1 | # Set larger code cache with -XX:ReservedCodeCacheSize=
graylog_1 | # JVM is running with Unscaled Compressed Oops mode in which the Java heap is
graylog_1 | # placed in the first 4GB address space. The Java Heap base address is the
graylog_1 | # maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress
graylog_1 | # to set the Java Heap base and to place the Java Heap above 4GB virtual address.
graylog_1 | # This output file may be truncated or incomplete.
graylog_1 | #
graylog_1 | # Out of Memory Error (workerManager.hpp:70), pid=7, tid=7
from graylog-docker.
pschichtel
commented on June 12, 2024
What @coffee-squirrel said, except that we have also noticed this behavior with alpine based temurin images.
from graylog-docker.
mpfz0r
commented on June 12, 2024
@MahdiGhiasi
I still have no idea what's causing this, but out of the blue, could you try this for me?
entrypoint: "/bin/cat /etc/profile.d/graylog.sh"
from graylog-docker.
mpfz0r
commented on June 12, 2024
@pschichtel Thanks! That makes a lot of sense. For reference, it's this comment: adoptium/containers#215 (comment) in particular
OK, so I guess we can close this case. Unless we need to support docker versions older than 20.10.10
from graylog-docker.
MahdiGhiasi
commented on June 12, 2024
I can confirm that updating docker to version 20.10.17 solves both issues we were facing.
from graylog-docker.
mpfz0r
commented on June 12, 2024
Great. I'm gonna keep this open in case more people are running into this.
In the worst case we could switch to the temurin focal docker images.
from graylog-docker.
Related Issues (20)
- Docker release job: Intermittent failure HOT 3
- Prometheus config file a bit misleading HOT 4
- Graylog config files should be sourced from corresponding release
- Updating to 4.3.4 or later causes a core dump HOT 3
- Migration from ElasticSearch to OpenSearch not working HOT 2
- health check for self-signed certificate still fails HOT 1
- Image vulnerablities HOT 18
- source address is always fill with docker internal IP HOT 1
- ps HOT 1
- /plugins-defualt/ and GRAYLOG_HOME HOT 7
- MongoDb 5 Docker failed to start after Upgrade HOT 6
- graylog.conf is not updated from env variables HOT 4
- Setting Graylog uid/gid as arguments in the docker build instead of being hard coded in the Dockerfile. HOT 1
- Docker entrypoint is inefficient with disk space HOT 1
- Docker entrypoint checks ownership on the wrong path HOT 1
- Remove graylog.conf copy for Graylog 6.0
- Swarm - Docker image healthcheck never passes HOT 6
- Unable to bind to netmaker ip
- API Web Portal Does not Redirect to Login
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from graylog-docker.