stardog-union / helm-charts Goto Github PK

The timeout for the alive check is 1s, which is generally fine, however when a pod is under higher load it can take longer than 1s to respond. We can better support more resource intensive workloads without causing pods to be killed prematurely by k8s by increasing the timeout.

Currently we deploy stardog into the helm release namespace. However, for users that have stardog as a dependency to another application and want it to run in its own namespace, they need to be able to override which namespace Stardog is deployed into so we should allow that via an option that can be set.

On first install, the post-install job sets the admin password for Stardog. This password is echoed to the standard out because the set -x builtin is enabled when the change_pw function is called. This should be removed because of the possibility of the password being swept up in logs.

Errors on pod when setting cluster up

NAME                        READY   STATUS             RESTARTS       AGE
pod/dogtest-stardog-0       0/1     CrashLoopBackOff   13 (33s ago)   42m
pod/dogtest-stardog-1       0/1     Init:0/1           0              42m
pod/dogtest-stardog-2       0/1     Init:0/1           0              42m
pod/dogtest-stardog-b5ccp   0/1     Error              0              42m
pod/dogtest-stardog-d5w6k   0/1     Error              0              42m
pod/dogtest-stardog-fsp9n   0/1     Error              0              26m
pod/dogtest-stardog-gdpjh   0/1     Error              0              38m
pod/dogtest-stardog-jb8g7   0/1     Error              0              41m
pod/dogtest-stardog-qmc46   0/1     Error              0              41m
pod/dogtest-stardog-wcxh4   0/1     Error              0              32m

❯ k logs pod/dogtest-stardog-b5ccp
+ function wait_for_start {
/bin/sh: 3: function: not found

Right now the timing for liveness and rediness probes is hard coded into the helm charts. When running on different clouds it can be useful to modify those parameters.

When using the default configuration, stardog does not have the permission to write ont he persistent volume. This is because the persistent volume is owned by root and stardog doesn't use run with the root user by default.

STARDOG_HOME '/var/opt/stardog/' is not writeable by the current user

A workaround is to run stardog as root:

  securityContext:
    enabled: true
    runAsUser: 0
    runAsGroup: 0
    fsGroup: 0

A better solution would be to use the busybox init container, not only when the cluster is enabled, to change the permissions before stardog starts.

We should make the image repository and pull policy for for busybox something that is settable. This is useful when wanting to use only local image repositories and not docker hub.

Hi,

I see that the default storage class value is standard, but it's not always there. For example on K3S the default storage class is named local-path.

I think it should perhaps be an empty string by default, to let kubernetes select the default storage class.

Currently the stardog pod tries to set the password every time it comes up, however, the password should only be changed once from the default in the Helm post install hook.

We provide values file variable tmpDir provide a mechanism to set the location of the java.io.tmpdir . However if folder doesn't exist it will fail. This is problematic in the case of wanting the temp folder to reside on the same place as stardog home because we have a newly created PVC disk and no way to create the folder in advance

The Stardog chart currently depends on the ZooKeeper chart in Helm Hub's incubator. With Helm 3 those charts are now being deprecated:

https://github.com/helm/charts/blob/master/README.md#deprecation-timeline

We should move to another actively maintained ZooKeeper chart if possible, such as:

https://github.com/bitnami/charts/tree/master/bitnami/zookeeper

Or if that's not possible, fork and support the incubated chart.

We would like to provide some basic ingress support for our helm charts for those who don't' wish to define ingress separately.

Hi, I've tried a PR but the test is failing, not sure why as this wasn't changed. Could someone take a look?

#64

Thanks,

D

We would like to have the capability to add tolerations to the Stardog pods, in case we want to deploy them into tainted nodes.

We are using stardog application- URL through virtual service deployment in our project, it was working properly but suddenly went down in all environments.

Hello, I have deployed Stardog in Kubernetes cluster, and to reduce idle resource usage I reduced cpu request to 0.2.

Java args used javaArgs: "-Xmx2g -Xms2g -XX:MaxDirectMemorySize=1g -XX:+UseContainerSupport -XX:MaxRAMPercentage=80 -XX:ActiveProcessorCount=2"

Stardog detects 0.2 CPU and fails to start, overrides from JVM args does not work, here is example

/opt/stardog/bin/stardog-admin server start --foreground --port 5820 --home /var/opt/stardog/ -XX:+UseContainerSupport -XX:MaxRAMPercentage=80 -XX:ActiveProcessorCount=2
Improperly specified VM option 'ActiveProcessorCount=0.2'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

If I unset the cpu request, I got empty ActiveProcessorCount

/opt/stardog/bin/stardog-admin server start --foreground --port 5820 --home /var/opt/stardog/ -XX:+UseContainerSupport -XX:MaxRAMPercentage=80 -XX:ActiveProcessorCount=2
Improperly specified VM option 'ActiveProcessorCount='
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Maybe there is another workaround ?

Reference Java doc: https://www.oracle.com/java/technologies/javase/8u191-relnotes.html

There is no way in the helmchart to add additional init-containers.

This could be useful for:

• Automatically executing stardog-admin commands to create an initial setup
• add jdbc drivers

A new field in the helmchart named "init-containers" would solve this.

Include a section in the helm charts where key/values would have the same stardog.properties effect. It would be nice if there was some section in the values that influenced the config map bit so only section would need to be updated.

We currently run a small set of tests with minikube on Circle, however, in order to provide checks that the charts work with the major k8s providers we should also integrate tests that run against:

• GKE
• AKS
• EKS

If your stardog.properties file is completely empty, the helm chart will fail to deploy, i think it expects at least 1 key=value pair. The exact error I get is

Error: YAML parse error on novacontext-chart/charts/stardog/templates/configmap.yaml: error converting YAML to JSON: yaml: line 14: did not find expected key

We discovered this as a result of a postprocessor we have that inserts a different stardog.properties file at the last minute, the one we inserted was blank, if we insert one with a random pair "key=value" it works. if blank, it fails, if we delete it at the last minute, it also fails with the same error

We should add a config options stardogArgs or something to allow additional flags to be passed to Stardog's server start.

When starting Stardog Cluster on existing volumes it's possible to encounter the following error:

Cannot start kernel since there are no other nodes in the cluster to sync up

This happens because the latest tx id is stored in ZK and the Stardog cluster node with a matching tx id needs to start first. With k8s this can be problematic because the default nature of statefulsets starts them in an ordered fashion so if the first node doesn't match the tx id then the deployment hangs and no other pods are deployed.

The setting podMangementPolicy: Parallel can be set on the Stardog statefulset to start all of the Stardog pods at the same time, which allows the pod with the matching tx id to start and the other nodes to sync.

We should make this a configurable option.

Hi another start item

`kubectl -n marsel create secret generic stardog-license --from-file stardog-license-key.bin=./stardog-license-key.bin
error: failed to create secret Post "http://localhost:8080/api/v1/namespaces/marsel/secrets?fieldManager=kubectl-create":

dial tcp [::1]:8080: connect: connection refused
`

Should there be some type of initiation ahead of time of kubectl?

8080 port is free

% lsof -i -P -n | grep 8080 %

As described here, the v1 of the Dockerhub API is deprecated and is being removed on September 5th. The URLs in the helm charts must be changed to the v2 versions to prevent issues.

The post install job still uses release name -stardog instead of the templated value

Hey,

as Istio requires strict naming, the name "sql" in the service for the port 5806/TCP is wrong.
To ensure Istio compatibility the name should be "mysql". Otherwise connections can't be established.

Cheers :)

Depending on the resources and where the charts are deployed Stardog can take awhile before it is completely deployed. We should tune any of the options that we can (and remove any unneeded sleeps, etc) to make this faster.

Bitnami has a new policy( bitnami/charts#10539) that they will no longer be posting references to charts older then six months in their main helm repository index file. They will continue to host the helm artifacts but you will need to obtain charts older then six months from previous versions of the index file. This breaks our current stardog chart that depends on zookeeper chart version 5.5.1.

We are working on a fix.

If there is an existing PVCs in a namespace when a brand new install is run. The password change post install job will fail because the password is already changed. This will fail the whole deployment.

The main problem is that a failed helm deployment doesn’t allow upgrades, which means we need to completely remove the deployed chart and install it again.

I used a randomly generated password that was not accepted. I used instead only ascii letters and it worked.

Add an option to disable the cluster and only deploy a single Stardog, no ZK.

The g1 gc has shown to be better than the current default UseParallelOldGC at minimizing lengthy gc pauses, which can lead to alive checks failing if the gc pauses exceed the timeout for alive checks.

Hi stardog

I am trying to do the kubctl setup I am running into some issues. I am sure I am missing something trivial

% git clone https://github.com/stardog-union/helm-charts stardog-helm

% ls -lrt
total 40
-rw-r--r--  1   staff   1763 Aug 24 08:13 CHANGELOG.md
-rw-r--r--  1   staff  11356 Aug 24 08:13 LICENSE
-rw-r--r--  1   staff   1956 Aug 24 08:13 README.md
drwxr-xr-x  3   staff     96 Aug 24 08:13 charts
drwxr-xr-x  4   staff    128 Aug 24 08:13 tests


% kubectl version
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.3", GitCommit:"ca643a4d1f7bfe34773c74f79527be4afd95bf39", GitTreeState:"clean", BuildDate:"2021-07-15T21:04:39Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"darwin/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?

% kubectl -n marselgraphs create secret generic stardog-license --from-file stardog-license-key.bin=.                               
error: cannot give a key name for a directory path

The post install job waits 600s for Stardog to start (including deploying ZK), this should be configurable in the values.yaml

Hi, been trying to configure this using Helm. The clustering does not work. I've re-installed several times now, but it never creates a cluster. The zookeeper instance set themselves up but running the stardog-admin cluster info command responds with "Not Found!"

This is the generated config file:

logging.audit.type=text
pack.enabled=true
pack.zookeeper.address=stardog-zookeeper-0.stardog-zookeeper-headless.stardog-kube:2181,stardog-zookeeper-1.stardog-zookeeper-headless.stardog-kube:2181,stardog-zookeeper-2.stardog-zookeeper-headless.stardog-kube:2181
pack.node.join.retry.count=15
pack.node.join.retry.delay=1m

If I check the logs one of the nodes always says sh: bad number for the zk start node and the output of the log is a single line

+ /opt/stardog/bin/stardog-admin server start --foreground --port 5820 --home /var/opt/stardog/

Is that normal?

Any help appreciated