Official Helm chart about docker

[jptissot]

It's for internal deployment for now. Any feedback is appreciated.

https://github.com/jptissot/matomo-chart

[vfbsilva]

@pierreozoux any updates on this issue?

[tburschka]

I'm also working on a helm chart: https://gitlab.com/ideaplexus/helm/matomo

Features:

• using matomo with apache
• using mariadb with replication
• support for traefik 2
• using configmap for config

Planned/Not working yet

• using redis for QueuedTracking
• config can only enabled in second deploy, since there is (currently) no way to create tables / default user / site by cli
• archive cronjob
• automatic download of maxmid geoip database with cronjob

PS: Documentation is coming, feedback would be grateful.

[kalib]

@appinteractive Sorry for the delay.. Just super busy at work these days. So, I got it working and pushed to our fork here: https://github.com/thinkresearch/charts/tree/MATOMO/incubator/matomo

IMPORTANT: Didn't have time to create the README.md yet, but plan to do so during the next couple of hours/days.

Will go over the code again and see if I can refactor it, change something for best practices, etc.. before sending a PR to the official helm charts repo.

Please, let me know if you have any thoughts.

[appinteractive]

Cool thank you so much! 😁

[jptissot]

I saw that bitnami has a docker image for matomo. They provide quite a few quality charts / images. Maybe worth a look: https://github.com/bitnami/bitnami-docker-matomo

[maximegaillard]

I follow this issue for some time now: we don't have an official chart but multiple custom charts. I think we all need an official chart, to at least get a repo for everyone contributions (including mine). If not, we’ll get a new repo each month 😟

[pierreozoux]

@kalib feel free to PR ;)

[appinteractive]

@kalib whats the status?

[kalib]

@appinteractive Yeah.. I got it working locally and in a gcp cluster for my tests. I just forked the original helm/charts repo today and will include my changes there and will let you know here...
After that, I'll just need to refactor the code a little bit, include a better readme file, etc.. before creating a pull request in their helm/charts repo. ;]

[vfbsilva]

@pierreozoux I do have it. I cant publish the files as they belong to the company I work for. But I can help you with a setup.

[pierreozoux]

Can't you ask your company to release it open source?

You got the Dockerfile for free :)

I know how to do it, I already did Rocket.Chat, Mautic, Jitsi.
I think it is nice to collaborate :)

But at the end, it is up to your company, indeed.

Tell them that the work to maintain it will be spread across various people and at the end of the day, it will be less expensive for your company. + it will give some datalove to your company, they can then say that they do open source. It is good to recruit devs/ops :)

[vfbsilva]

I work for the Goverment it will take like 6 months to get an approval if so. But I can take a look on the data I have and see what can I release. Do you have already a kubernetes cluster? The installation was quite straightforward, so maybe I can help you with the service, ingress and networkpolice? Is that what you need?

[maikotz]

The installation on k8s is indeed quite straight forward, just some issues with the latest 3.0.3 update.
I'm working on it and will report back here.

[maikotz]

Here is my running config, unfortunately there are two little issues with it:

1. piwik config file is hardcoded with usernames and passwords for mail host and database. This should be done with an initContainer which generates the piwik config file from the database Secret and configMap. Just hadn't had time to do it.
2. Since I need more memory for PHP I overwrite the shipped php.ini and restart the php worker processes in the fpm container. Not really a clean solution but WFM.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: piwik
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      name: piwik
      labels:
        app: piwik
        role: frontend
    spec:
    # Unfortunately not in 1.5 yet 
    # This is here for future correction as soon as cluster is on 1.6 or above
    # # These containers are run during pod initialization
    #   initContainers:
    #   - name: install
    #     image: busybox
    #     command:
    #     - sh
    #     - -c
    #     - cp /etc/config/config.ini.php /var/www/html/config/
    #     volumeMounts:
    #     - name: piwik-conf
    #       mountPath: "/etc/config/"
      containers:
      - image: piwik:3.0.3-fpm
        name: piwik-php
        lifecycle:
          postStart:
            exec:
              command:
                - /bin/sh
                - -c                
                - cp /etc/config/config.ini.php /var/www/html/config/; cp /etc/config/php.ini /usr/local/etc/php/php.ini; kill -USR2 1
        ports:
        - containerPort: 9000
          name: piwik-port
        volumeMounts:
        - name: html-dir
          mountPath: /var/www/html
        - name: piwik-conf
          mountPath: /etc/config/
      - image: nginx:1.11.12
        name: piwik-proxy
        args: ["nginx", "-g", "daemon off;", "-c", "/etc/config/nginx.conf"]
        ports:
        - containerPort: 80
          name: http
        volumeMounts:
        - name: piwik-nginx-conf
          mountPath: /etc/config/
        - name: html-dir
          mountPath: /var/www/html
      volumes:
      - name: piwik-nginx-conf
        configMap: 
          name: piwik-nginx-conf
      - name: piwik-conf
        configMap: 
          name: piwik-conf
      - name: html-dir
        emptyDir: {}
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: piwik-nginx-conf
data:
  nginx.conf: |-
    user nginx;
    worker_processes  1;

    error_log  /var/log/nginx/error.log warn;
    pid        /var/run/nginx.pid;

    events {
      worker_connections 1024;
    }

    http {
      upstream backend {
        server localhost:9000;
      }

      include /etc/nginx/mime.types;
      default_type application/octet-stream;
      gzip on;
      gzip_disable "msie6";
      
      server {
        listen 80;

        root /var/www/html/;
        index index.php index.html index.htm;

        location / {
          try_files $uri $uri/ =404;
        }

        error_page 404 /404.html;
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
          root /usr/share/nginx/html;
        }

        location = /favicon.ico {
          log_not_found off;
          access_log off;
        }
       
        location ~ \.php$ {

          fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
          fastcgi_param  SERVER_SOFTWARE    nginx;
          fastcgi_param  QUERY_STRING       $query_string;
          fastcgi_param  REQUEST_METHOD     $request_method;
          fastcgi_param  CONTENT_TYPE       $content_type;
          fastcgi_param  CONTENT_LENGTH     $content_length;
          fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
          fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
          fastcgi_param  REQUEST_URI        $request_uri;
          fastcgi_param  DOCUMENT_URI       $document_uri;
          fastcgi_param  DOCUMENT_ROOT      $document_root;
          fastcgi_param  SERVER_PROTOCOL    $server_protocol;
          fastcgi_param  REMOTE_ADDR        $remote_addr;
          fastcgi_param  REMOTE_PORT        $remote_port;
          fastcgi_param  SERVER_ADDR        $server_addr;
          fastcgi_param  SERVER_PORT        $server_port;
          fastcgi_param  SERVER_NAME        $server_name;
          fastcgi_intercept_errors on;
          fastcgi_pass backend;
        }
      }
    }
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: piwik-conf
data:
  php.ini: |-
    always_populate_raw_post_data=-1
    extension=geoip.so
    geoip.custom_directory=/var/www/html/misc
    memory_limit=256M

  config.ini.php: |-
    ; <?php exit; ?> DO NOT REMOVE THIS LINE
    ; file automatically generated or modified by Piwik; you can manually override the default values in global.ini.php by redefining them in this file.
    [database]
    host = "backend-sql"
    username = "***"
    password = "***"
    dbname = "***"
    tables_prefix = "piwik_"

    [General]
    proxy_client_headers[] = "HTTP_X_FORWARDED_FOR"
    proxy_host_headers[] = "HTTP_X_FORWARDED_HOST"
    salt = "***"
    trusted_hosts[] = "***"
    session_save_handler = dbtable
    minimum_memory_limit = -1

    [mail]
    transport = "smtp"
    port = "25"
    host = "***"
    type = "Plain"
    username = "***"
    password = "***"
    encryption = "tls"

    [Plugins]
    Plugins[] = "CorePluginsAdmin"
    Plugins[] = "CoreAdminHome"
    Plugins[] = "CoreHome"
    Plugins[] = "WebsiteMeasurable"
    Plugins[] = "Diagnostics"
    Plugins[] = "CoreVisualizations"
    Plugins[] = "Proxy"
    Plugins[] = "API"
    Plugins[] = "ExamplePlugin"
    Plugins[] = "Widgetize"
    Plugins[] = "Transitions"
    Plugins[] = "LanguagesManager"
    Plugins[] = "Actions"
    Plugins[] = "Dashboard"
    Plugins[] = "MultiSites"
    Plugins[] = "Referrers"
    Plugins[] = "UserLanguage"
    Plugins[] = "DevicesDetection"
    Plugins[] = "Goals"
    Plugins[] = "Ecommerce"
    Plugins[] = "SEO"
    Plugins[] = "Events"
    Plugins[] = "UserCountry"
    Plugins[] = "VisitsSummary"
    Plugins[] = "VisitFrequency"
    Plugins[] = "VisitTime"
    Plugins[] = "VisitorInterest"
    Plugins[] = "ExampleAPI"
    Plugins[] = "RssWidget"
    Plugins[] = "Feedback"
    Plugins[] = "Monolog"
    Plugins[] = "Login"
    Plugins[] = "UsersManager"
    Plugins[] = "SitesManager"
    Plugins[] = "Installation"
    Plugins[] = "CoreUpdater"
    Plugins[] = "CoreConsole"
    Plugins[] = "ScheduledReports"
    Plugins[] = "UserCountryMap"
    Plugins[] = "Live"
    Plugins[] = "CustomVariables"
    Plugins[] = "PrivacyManager"
    Plugins[] = "ImageGraph"
    Plugins[] = "Annotations"
    Plugins[] = "MobileMessaging"
    Plugins[] = "Overlay"
    Plugins[] = "SegmentEditor"
    Plugins[] = "Insights"
    Plugins[] = "Morpheus"
    Plugins[] = "Contents"
    Plugins[] = "BulkTracking"
    Plugins[] = "Resolution"
    Plugins[] = "DevicePlugins"
    Plugins[] = "Heartbeat"
    Plugins[] = "Intl"
    Plugins[] = "Marketplace"
    Plugins[] = "ProfessionalServices"
    Plugins[] = "UserId"
    Plugins[] = "CustomPiwikJs"
    Plugins[] = "Provider"

    [PluginsInstalled]
    PluginsInstalled[] = "Diagnostics"
    PluginsInstalled[] = "Login"
    PluginsInstalled[] = "CoreAdminHome"
    PluginsInstalled[] = "UsersManager"
    PluginsInstalled[] = "SitesManager"
    PluginsInstalled[] = "Installation"
    PluginsInstalled[] = "Monolog"
    PluginsInstalled[] = "Intl"
    PluginsInstalled[] = "CorePluginsAdmin"
    PluginsInstalled[] = "CoreHome"
    PluginsInstalled[] = "WebsiteMeasurable"
    PluginsInstalled[] = "CoreVisualizations"
    PluginsInstalled[] = "Proxy"
    PluginsInstalled[] = "API"
    PluginsInstalled[] = "ExamplePlugin"
    PluginsInstalled[] = "Widgetize"
    PluginsInstalled[] = "Transitions"
    PluginsInstalled[] = "LanguagesManager"
    PluginsInstalled[] = "Actions"
    PluginsInstalled[] = "Dashboard"
    PluginsInstalled[] = "MultiSites"
    PluginsInstalled[] = "Referrers"
    PluginsInstalled[] = "UserLanguage"
    PluginsInstalled[] = "DevicesDetection"
    PluginsInstalled[] = "Goals"
    PluginsInstalled[] = "Ecommerce"
    PluginsInstalled[] = "SEO"
    PluginsInstalled[] = "Events"
    PluginsInstalled[] = "UserCountry"
    PluginsInstalled[] = "VisitsSummary"
    PluginsInstalled[] = "VisitFrequency"
    PluginsInstalled[] = "VisitTime"
    PluginsInstalled[] = "VisitorInterest"
    PluginsInstalled[] = "ExampleAPI"
    PluginsInstalled[] = "RssWidget"
    PluginsInstalled[] = "Feedback"
    PluginsInstalled[] = "CoreUpdater"
    PluginsInstalled[] = "CoreConsole"
    PluginsInstalled[] = "ScheduledReports"
    PluginsInstalled[] = "UserCountryMap"
    PluginsInstalled[] = "Live"
    PluginsInstalled[] = "CustomVariables"
    PluginsInstalled[] = "PrivacyManager"
    PluginsInstalled[] = "ImageGraph"
    PluginsInstalled[] = "Annotations"
    PluginsInstalled[] = "MobileMessaging"
    PluginsInstalled[] = "Overlay"
    PluginsInstalled[] = "SegmentEditor"
    PluginsInstalled[] = "Insights"
    PluginsInstalled[] = "Morpheus"
    PluginsInstalled[] = "Contents"
    PluginsInstalled[] = "BulkTracking"
    PluginsInstalled[] = "Resolution"
    PluginsInstalled[] = "DevicePlugins"
    PluginsInstalled[] = "Heartbeat"
    PluginsInstalled[] = "Marketplace"
    PluginsInstalled[] = "ProfessionalServices"
    PluginsInstalled[] = "UserId"
    PluginsInstalled[] = "CustomPiwikJs"
    PluginsInstalled[] = "Provider"

---

[vfbsilva]

@maikotz assuming we are discussing a solution on the context of 11 principles the file cannot be hard-coded this is exactly what we are trying to avoid.
@pierreozoux I might have missed the discussion but we seek to provide the configuration data as parameters to the deploy right?

[vfbsilva]

@pierreozoux here goes, ive removed some sensitive data.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: piwiki
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: piwiki
    spec:
      containers:
        - name: piwiki
          image: @TODO: REMOVED
          imagePullPolicy: Always
          ports:
            - containerPort: 80
          resources:
            limits:
              memory: 2Gi

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/rewrite-target: /
    ingress.kubernetes.io/whitelist-source-range: 10.0.0.0/8,127.0.0.1/32
  creationTimestamp: 2017-03-13T18:37:30Z
  generation: 2
  name: piwik
  namespace: @TODO: ADD HERE
  resourceVersion: "19307725"
spec:
  rules:
  - host: @TODO: ADD HERE
    http:
      paths:
      - backend:
          serviceName: piwik-1774830804-zmx1s
          servicePort: 80
        path: /
  tls:
  - hosts:
    - @TODO: ADD HERE
status:
  loadBalancer: {}

kind: NetworkPolicy
apiVersion: extensions/v1beta1
metadata:
  name: piwiki 
spec:
  podSelector:
    matchLabels:
      app: piwiki 
  ingress:
    - ports:
      - protocol: TCP
        port: 80
      - protocol: TCP
        port: 443

So what I would expect is that this section from config.ini.php to be configured through ENV vars:

[database]
host = "@hostname"
username = "@USERNAME"
password = "@password"
dbname = "DNAME"
tables_prefix = "blah_"
charset = "utf8"

So I could invoke the container like:
kubectl run --HOSTANAME="blah" --USERNAME="blah" --PASSWORD="blah" --DBNAME="blah"-i -t piwiki.image:latest /bin/bash

[maikotz]

@vfbsilva you're right it shouldn't be hardcoded, especially with the plugin list and everything else but it seems that piwik itself doesn't support it so far, there are other open issues and forum posts regarding the issue with hardcoded configs.

Issue #10914
Piwik Forum

My suggestion would be an init-container that reads the environment variables and creates the config on-demand by replacing placeholders in the config.sample file

[vfbsilva]

@maikotz I'm sorta new to the community and I do not want to start a flamme war so if you think it is better we can discuss it via mail. As long it is oki and as I find there is a misunderstanding regarding our expectations I will explain my view here.
TL;DR; It does not support.

Problem, Discussion, Expectation
Problem:
Ethereal file systems: the data is created and destroyed within the app space (pod, container whatever), I need the deploy to be resilient. Ie: I've crashed a frontend, I've added another one, I want to be aple to give the database config as paramethers, hence I can have 1 job which makes all my deploys from a auth list for example.

Discussion:
I don't know how an init container would work. But I ask, if I have multiple piwiks in a single namespace will this init container work for this kind of setup?

Culprit database section from config file:
[database]
host = "@hostname"
username = "@USERNAME"
password = "@password"
dbname = "DNAME"
tables_prefix = "blah_"
charset = "utf8"

Possible Solution(Expected behaviour, assuming Kubernets):
The database config belongs to a pod. Not to a service or deployment, it belongs to the pod itself. How do I invoke a pod:
kubectl run -i -t piwiki.image:latest /bin/bash

How could I possible provide the info to the pod?
kubectl run --HOSTANAME="blah" --USERNAME="blah" --PASSWORD="blah" --DBNAME="blah"-i -t piwiki.image:latest /bin/bash

How are enviroment variables supported on Kubernets:
https://kubernetes.io/docs/tasks/configure-pod-container/define-environment-variable-container/

What I would expect to generate with my command:

apiVersion: v1
kind: Pod
metadata:
  name: envar-demo
  labels:
    purpose: demonstrate-envars
spec:
  containers:
  - name: envar-demo-container
    image: gcr.io/google-samples/node-hello:1.0
    env:
    - name: HOSTNAME
      value: "blah"
   - name: DBNAME
     value: "blah"
   - name: PASSWORD
    value: "blah"
   - name: USERNAME
    value: "blah"

So no need of an init container IMHO. You could have a "sed/awk" script which replaces the config within the file but this is a hack not something really supported and would break on update. but @pierreozoux needed those files to run the image on kubernets.

@maikotz further reference: https://12factor.net/config

[martinszy]

A helm chat would be good, has anybody got one working?

[pierreozoux]

Sorry, priority shifted.. anybody, feel free to PR the official helm chart :)

[kalib]

Any plans on moving the Helm chart you have hosted at https://github.com/matomo-org/docker/tree/master/.examples/helm to the official helm charts repository? At least to the incubator one? https://github.com/helm/charts

[Findus23]

@pierreozoux Quick question about

docker/.examples/helm/templates/configmap.yaml

Line 11 in 705284f

nginx.conf: |

(Disclaimer: I know nothing about kubernetes and co.)

Does the linked nginx config use nginx as a reverse proxy or a primary webserver? Because in the latter case, Matomo wouldn't be secure by default as the .htaccess files are not read and therefore e.g. tmp/ would be public)

Maybe this should be compared with https://github.com/matomo-org/matomo-nginx

[kalib]

@kalib feel free to PR ;)

Cool, will make a few changes and will do. thanks.

[kalib]

@appinteractive no problem.. will keep you in the loop.

[appinteractive]

@kalib so still not success?

[jptissot]

Well, I just successfully built a basic helm chart that uses the bitnami docker image. I might share it if someone is interested.

[appinteractive]

@jptissot sure, are you planing to use it in production?

[veerappans]

Hello Guys, I am facing the issue where whenever my pod restarts it looses DB config. I am using the helm charts from ".example" folder. I see a lot of discussion here for initializing the DB config, but I am not clear on the solution. Can someone please help me with config. For some reason, the pod keeps crashing and it stops tracking. And we dont notice it for a day or two and loose usage as it needs to be set up again. Still need to figure out why it keeps crashing. But at least for next time, we wont loose data.

[kalib]

@kalib so still not success?

Sorry @appinteractive I no longer use Matomo... :/ I am no longer working on that.

[dominik-bln]

Hello Guys, I am facing the issue where whenever my pod restarts it looses DB config. I am using the helm charts from ".example" folder. I see a lot of discussion here for initializing the DB config, but I am not clear on the solution. Can someone please help me with config. For some reason, the pod keeps crashing and it stops tracking. And we dont notice it for a day or two and loose usage as it needs to be set up again. Still need to figure out why it keeps crashing. But at least for next time, we wont loose data.

Sounds and looks like the example Chart is missing a volume for the created configuration, i. E. the data is written inside the running container to some ephemeral location and then lost on restart.

The chart linked in the comment above is working for us in Production with some minor inconveniences (I.e missing cronjob and config from configmap would be nicer).

[softatac]

Hi, any news on this?
Considering as an option to use in a k8s env, but not sure which chart to choose from

[tburschka]

Hi, any news on this?
Considering as an option to use in a k8s env, but not sure which chart to choose from

Hi, thanks for the reply, i was busy for now, but i will try to update the chart to the latest matomo version.

[adlord]

Hi, any news on this?
Considering as an option to use in a k8s env, but not sure which chart to choose from

Hi, thanks for the reply, i was busy for now, but i will try to update the chart to the latest matomo version.

Hello. I'm also really interrested in your helm chart !

[kevin-shelaga]

I built off of the very fine work @tburschka did and have a working copy in github for people to use. I also have a working copy locally with cloud sql mysql, would like to build that in too. https://github.com/kevin-shelaga/matomo-helm-chart

[tburschka]

@kevin-shelaga: mine should work too. But i added your update on the README.md and Notes.txt. Thank you for that.

I've pushed the latest changes. (sorry, I was lazy on that).

notable changes

• added persistence
• redis is now standalone
• disable debug for mariadb
• updated secret generation based on values/config
• added cronjob (requires RWX as storage) => not yet configurable/must be disabled when storage is RWO

[kevin-shelaga]

Thanks @tburschka, there was also a bug here https://github.com/kevin-shelaga/matomo-helm-chart/blob/46b55b808820126d8a757a5c7ac214fb2d9e6049/templates/service.yaml#L11

[tburschka]

@kevin-shelaga I will check it

[tburschka]

@kevin-shelaga That's not a bug, that's a feature...

https://kubernetes.io/docs/concepts/services-networking/service/

Port definitions in Pods have names, and you can reference these names in the targetPort attribute of a Service. This works even if there is a mixture of Pods in the Service using a single configured name, with the same network protocol available via different port numbers. This offers a lot of flexibility for deploying and evolving your Services. For example, you can change the port numbers that Pods expose in the next version of your backend software, without breaking clients.

It should not be a concern of the Service what port your deployment is using. So this alias ensures, when you change the port that the service does not need to be changed.

[kevin-shelaga]

Interesting, maybe it was the port change from 80 to 8080 that was the real issue then.

Have you tested the persistence yet? Running into issues with permissions on mkdir. No cronjob. Just persistence for matomo.

[tburschka]

As far as i know i hadn't. But i can test it with an secondary instance...

[kevin-shelaga]

@tburschka getting mkdir: cannot create directory '/bitnami/matomo': Permission denied with persistence enabled and no cronjobs, 1 replica on gke

needed to set

podSecurityContext:
  fsGroup: 2000

[tburschka]

I will update the docs....

[wethinkagile]

Do we have an offical chart yet? Bitnami charts are nice, but from the vendor a helm chart should be provided. Thanks

[kantum]

Today there are 5 results on artifacthub for matomo but none of them seems to work out of the box.

I get Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "Ingress" in version "networking.k8s.io/v1beta1"

I guess this is not a big deal, but I do not have time to dig into that, I was just looking for a fast way to test the solution.

[kevin-shelaga]

@kantum you get that error because youre probably running K8s 1.22 and the charts arent updated to use the newest api version, due to the deprecations K8s 1.22

[tburschka]

@kantum have you tried mine? https://gitlab.com/ideaplexus/helm/matomo I know i still lack on add it to the artifact hub. If there is any issue, just write me here or create a ticket...

[mattab]

FYI also noticed there seems to be one here: https://github.com/bitnami/charts/tree/main/bitnami/matomo#bitnami-package-for-matomo
Not sure if this is useful for your project @tburschka and/or for anyone else looking for a Helm Chart for Matomo to deploy and manage Matomo on Kubernetes clusters?

[tburschka]

@mattab I've noticed this Helm chart, but there were some limitations at this time. If i see it correct it's still not possible to download GeoIP Database with this chart. But I'm not into it since I've currently no time for this project. The idea was to create a small Go application which does the bootstrapping (plugins, geoip-db, permissions)... and use this as init container in my chart . But as I'm said, I'm currently not on active development for this :-(