jimdo / prometheus_client_php

Prometheus instrumentation library for PHP applications

❗ New project maintainers ❗

This project is no longer maintained here. Please go to https://github.com/endclothing/prometheus_client_php.

A prometheus client library written in PHP

This library uses Redis or APCu to do the client side aggregation. If using Redis, we recommend to run a local Redis instance next to your PHP workers.

How does it work?

Usually PHP worker processes don't share any state. You can pick from three adapters. Redis, APC or an in memory adapter. While the first needs a separate binary running, the second just needs the APC extension to be installed. If you don't need persistent metrics between requests (e.g. a long running cron job or script) the in memory adapter might be suitable to use.

Usage

A simple counter:

\Prometheus\CollectorRegistry::getDefault()
    ->getOrRegisterCounter('', 'some_quick_counter', 'just a quick measurement')
    ->inc();

Write some enhanced metrics:

$registry = \Prometheus\CollectorRegistry::getDefault();

$counter = $registry->getOrRegisterCounter('test', 'some_counter', 'it increases', ['type']);
$counter->incBy(3, ['blue']);

$gauge = $registry->getOrRegisterGauge('test', 'some_gauge', 'it sets', ['type']);
$gauge->set(2.5, ['blue']);

$histogram = $registry->getOrRegisterHistogram('test', 'some_histogram', 'it observes', ['type'], [0.1, 1, 2, 3.5, 4, 5, 6, 7, 8, 9]);
$histogram->observe(3.5, ['blue']);

Manually register and retrieve metrics (these steps are combined in the getOrRegister... methods):

$registry = \Prometheus\CollectorRegistry::getDefault();

$counterA = $registry->registerCounter('test', 'some_counter', 'it increases', ['type']);
$counterA->incBy(3, ['blue']);

// once a metric is registered, it can be retrieved using e.g. getCounter:
$counterB = $registry->getCounter('test', 'some_counter')
$counterB->incBy(2, ['red']);

Expose the metrics:

$registry = \Prometheus\CollectorRegistry::getDefault();

$renderer = new RenderTextFormat();
$result = $renderer->render($registry->getMetricFamilySamples());

header('Content-type: ' . RenderTextFormat::MIME_TYPE);
echo $result;

Change the Redis options (the example shows the defaults):

\Prometheus\Storage\Redis::setDefaultOptions(
    [
        'host' => '127.0.0.1',
        'port' => 6379,
        'password' => null,
        'timeout' => 0.1, // in seconds
        'read_timeout' => 10, // in seconds
        'persistent_connections' => false
    ]
);

Using the InMemory storage:

$registry = new CollectorRegistry(new InMemory());

$counter = $registry->registerCounter('test', 'some_counter', 'it increases', ['type']);
$counter->incBy(3, ['blue']);

$renderer = new RenderTextFormat();
$result = $renderer->render($registry->getMetricFamilySamples());

Also look at the examples.

Development

Dependencies

• PHP 5.6
• PHP Redis extension
• PHP APCu extension
• Composer
• Redis

Start a Redis instance:

docker-compose up Redis

Run the tests:

composer install

# when Redis is not listening on localhost:
# export REDIS_HOST=192.168.59.100
./vendor/bin/phpunit

Black box testing

Just start the nginx, fpm & Redis setup with docker-compose:

docker-compose up

Pick the adapter you want to test.

docker-compose run phpunit env ADAPTER=apc vendor/bin/phpunit tests/Test/
docker-compose run phpunit env ADAPTER=redis vendor/bin/phpunit tests/Test/

prometheus_client_php's Issues

Same type metrics rendered twice

We have an issue with client rendering same metrics twice.

Using APC storage and "jimdo/prometheus_client_php": "^0.7.0".

The output contains:

# HELP config_error 
# TYPE config_error counter
.
.
.
# HELP config_error 
# TYPE config_error counter

Here's the php code for rendering:

<?php
require "vendor/autoload.php";

$adapter = new Prometheus\Storage\APC();
$registry = new \Prometheus\CollectorRegistry($adapter);
$renderer = new \Prometheus\RenderTextFormat();

$result = $renderer->render($registry->getMetricFamilySamples());

header('Content-type: ' . \Prometheus\RenderTextFormat::MIME_TYPE);
echo $result;

I noticed that print_r($metrics) in vendor/jimdo/prometheus_client_php/src/Prometheus/RenderTextFormat.php:22 also show the counter doubled.

REDIS EXPIRE key

Can you explain this?

Expose the metrics:

$registry = \Prometheus\CollectorRegistry::getDefault();
$renderer = new RenderTextFormat();
$result = $renderer->render($registry->getMetricFamilySamples());
header('Content-type: ' . RenderTextFormat::MIME_TYPE);
echo $result;

It means that every page call will show more and more counters?
Do i need flush redis after page call?

Use HTTPPLUG

As guzzle 6 can collide with other dependencies, the HTTP layer should use http://docs.php-http.org/en/latest/ HTTPPLUG with the discovery factory to use whatever HTTP client is present.

Php5.5 installed prometheus, encountered apc related issues.

When I install apcu 5.1.3 extension, add an APCU class to replace the APC class, such as modifying the apc_add to replace apcu_add, but after adding the count by example to get no metrics.

Replace/rewrite InMemory adapter

With #18 we removed all usages of the InMemory adapter.
Either remove it altogether or rewrite it.

No documentation?

There's no documentation as far as I can tell. I can't seem to find any.

How to install or use it in my own project?

Is there detailed documentation to guide me in installing and deploying this client?Thank you.

Project status?

Hello,

firstly thanks for good job with develop prometheus client implementation for PHP.

I want to ask this project is still maintained?

if labelvalues include ':', it will failed to render

If label values include ':', it will failed to render. But it could set labelvalues whit ':'(preg_match allowed ':').

Replace double data type with float data type

It looks like the library uses double keyword for a phpdoc comments in some places. PHP doesn't really differentiate between float and double with the former being much more common. IMHO it makes it confusing when reading the code. Additionally, some tools may not handle double data type correctly (for instance PHPStan). Therefore I would suggest switching double to float.

Forbid different label names for the same metric

As stated here:

Client libraries MUST NOT under any circumstances allow users to have different label names for the same metric for Gauge/Counter/Summary/Histogram or any other Collector offered by the library.

Should support ext-apcu instead of ext-apc

ext-apc is the old apc userland cache. Now its all in apcu

apcu support via APCUIterator?

Hello, I have php 7.1.13 with apcu which exposes APCUIterator, not APCIterator (line 198, Storage/APC.php) ... am I missing something? Your code crashes with class not found. I see that the dependency is on php 5.6, I assumed php 5.6 or higher?

Is the project still alive and supported?

or abandoned?

Allow setting value of counter metrics

I have the need for counter metrics that are set explicitly, instead of being incremented. I don't have access to the system tracking the metric, I only have whatever the current value is. One good example of this is to provide an uptime metric.

$u = getCurrentUptime();
$uptime = $registry->registerCounter($ns, 'uptime_seconds_total', 'Current uptime in seconds');
$uptime->set($u);

Incorrect name in Exception

When checking the label name it displays metricName on error.

diff --git a/src/Prometheus/Collector.php b/src/Prometheus/Collector.php
index 7efc5d6..edc5e80 100644
--- a/src/Prometheus/Collector.php
+++ b/src/Prometheus/Collector.php
@@ -32,7 +32,7 @@ abstract class Collector
         $this->help = $help;
         foreach ($labels as $label) {
             if (!preg_match(self::RE_METRIC_LABEL_NAME, $label)) {
-                throw new \InvalidArgumentException("Invalid label name: '" . $metricName . "'");
+                throw new \InvalidArgumentException("Invalid label name: '" . $label . "'");
             }
         }
         $this->labels = $labels;

Exception when sorting gauge values with labels in Redis.php

I'm just opening this as a reference to an upcoming PR.

I found an issue that happens when you sort gauge entries with labels. You json_decode the labels and expect them to work with them as an array but json decode produces objects by default. I also have a similar error with the TextFormatRenderer that expects arrays instead of stdclasses.

PR coming soon, by the end of day probably.

stream_set_timeout() expects parameter 1 to be resource, boolean given

Error with relevant slice of the stack trace:

exception 'ErrorException' with message 
  'stream_set_timeout() expects parameter 1 to be resource, boolean given'
  in /var/www/vendor/jimdo/prometheus_client_php/src/Prometheus/Storage/Redis.php:100
Stack trace:
  #0 (): Illuminate\Foundation\Bootstrap\HandleExceptions->handleError()
  #1 /var/www/vendor/jimdo/prometheus_client_php/src/Prometheus/Storage/Redis.php(100): Redis->setOption()
  #2 /var/www/vendor/jimdo/prometheus_client_php/src/Prometheus/Storage/Redis.php(142): Prometheus\Storage\Redis->openConnection()
  #3 /var/www/vendor/jimdo/prometheus_client_php/src/Prometheus/Gauge.php(31): Prometheus\Storage\Redis->updateGauge()
  #4 /var/www/app/Services/MetricsService.php(38): Prometheus\Gauge->set()

I don't know if this helps: https://stackoverflow.com/questions/22898810/cant-get-stream-set-timeout-to-work-in-foreach-array-loop

If you need any more information let me know, I don't really know anything about PHP but I can ask someone who does.

Redis::collectHistograms() throws PHP warning: sort() expects parameter 1 to be array, boolean given

When no histogram data have been pushed into redis yet, calling Prometheus\Storage\Redis::collectHistograms results in a PHP warning:

warning: sort() expects parameter 1 to be array, boolean given

This is because

$keys = $this->redis->sMembers(self::$prefix . Histogram::TYPE . self::PROMETHEUS_METRIC_KEYS_SUFFIX);

returns false.

APC(u) store – why not scrapbook?

Today I noticed you're working on storing data in APC(u). Did you look at https://github.com/mattiasmullie/scrapbook especially http://www.scrapbook.cash/interfaces/key-value-store/? It's very lightweight. It provides locking primitives, native float increments and lots of other backends.

Labels are creating duplicated metrics

I noticed that registering 2 different countrers (different namespaces and names) with 2 different labels creates a weird situation where metrics are messed up.

Here's how I register the 2 counters:

Counter 1

$counter = $this->metricsClient->getOrRegisterCounter(
    'oosm',
    'successful_login_total',
    'Total of successful login',
    ['role']
);

$roles = $event->getUser()->getRoles();
foreach ($roles as $role) {
    $counter->inc([$role]);
}

Counter 2

$counter = $this->metricsClient->getOrRegisterCounter(
    'oosm',
    'location_put_offline_total',
    'Total of location put offline',
    ['location_name']
);
$counter->inc([$event->getLocationName()]);

When I hit both this counters, the metric endpoint is returning:

# HELP oosm_location_put_offline_total Total of location put offline 
# TYPE oosm_location_put_offline_total counter 
oosm_location_put_offline_total{location_name="Springfield Avenue"} 1    <---- Ok

# HELP oosm_successful_login_total Total of successful login
# TYPE oosm_successful_login_total counter 

oosm_location_put_offline_total{role="Springfield Avenue"} 1    <---- This is wrong!
oosm_successful_login_total{role="ADMINISTRATOR"} 2    <---- Ok

Could be related to: #34 ?

PHP version: 7.1.13

Counters don't support float values

Great work! Noticed one issue though – It seems as if counters don't handle floats yet. The behavior of the redis adapter and the apc adapter differs: Redis completly ignores float values where apc counts their int values and discards the decimal:

Redis

~/s/prometheus_client_php ❯❯❯ docker-compose up -d
~ ❯❯❯ curl 'localhost:8080/examples/some_counter.php?adapter=redis&c=2.7'
OK
~ ❯❯❯ curl 'localhost:8080/examples/some_counter.php?adapter=redis&c=2.7'
OK
~ ❯❯❯ curl 'localhost:8080/examples/metrics.php?adapter=redis'

~ ❯❯❯

APC

~/s/prometheus_client_php ❯❯❯ docker-compose up -d
~/s/prometheus_client_php ❯❯❯ curl 'localhost:8080/examples/some_counter.php?adapter=apc&c=2.7'                                                        OK
~/s/prometheus_client_php ❯❯❯ curl 'localhost:8080/examples/some_counter.php?adapter=apc&c=2.7'                                                        OK
~/s/prometheus_client_php ❯❯❯ curl 'localhost:8080/examples/metrics.php?adapter=apc'                                                                   # HELP test_some_counter it increases
# TYPE test_some_counter counter
test_some_counter{type="blue"} 4

The metric should read 5.4 I guess.

# HELP test_some_counter it increases
# TYPE test_some_counter counter
test_some_counter{type="blue"} 5.4

Incorporate feedback from mailing list

In response to https://groups.google.com/forum/#!topic/prometheus-developers/AgnAsR5qDiw we got the following feedback:

On 4 July 2016 at 08:26, <[email protected]> wrote:

> Hi,
>
> we just released https://github.com/Jimdo/prometheus_client_php/.
>

I'd recommend reading
https://prometheus.io/docs/instrumenting/writing_clientlibs/ There's a few
things like label handling, method naming and use of floats that could be
improved.

>
> It's a PHP implementation of the Prometheus client library.
> It currently supports Counters, Gauges and Histograms. For expositing
> metrics we currently support the text/HTTP output.
>
> It should run on PHP>=5.3. For storing metrics in between requests, we
> chose to use Redis.
> It's fairly lightweight and easy to deploy. While it might seem a quite
> big dependency, we chose it for ease of implementation. Storing metrics in
> http://php.net/manual/en/book.apcu.php turned out to be cumbersome (not
> saying it couldn't be done).
>

Any network dependency is big, and makes things very complex. For example
you're presuming that Gauges are only set (
https://github.com/Jimdo/prometheus_client_php/blob/9e644b5ca7d9b91be58f6b9b121e5bef17a6b78f/src/Prometheus/RedisAdapter.php#L74)
which is not the only way they can be used.

Brian

Add support for UTF-8 characters for label values

According to the Prometheus documentation

label_value can be any sequence of UTF-8 characters

Expected behaviour

Label values can be any sequence of UTF-8 characters.

Current behaviour

No measures are taken to ensure that the label values can be persisted without affecting the retrieval process.

Steps to reproduce

1. Persist a metric (counter, gauge, histogram) using APC or In-Memory storage, which has a label with value : (semicolon).
2. Retrieve the metrics and observe PHP warnings and metrics with missing label.

<?php

require __DIR__ . '/vendor/autoload.php';

//$adapter = new Prometheus\Storage\APC();
//$adapter->flushAPC();

$adapter = new Prometheus\Storage\InMemory();
$adapter->flushMemory();

$registry = new Prometheus\CollectorRegistry($adapter);

$counter = $registry->registerCounter('test', 'some_counter', 'it increases', ['type']);
$counter->incBy(1, [':']);

$renderer = new Prometheus\RenderTextFormat();
$result = $renderer->render($registry->getMetricFamilySamples());

header('Content-type: ' . Prometheus\RenderTextFormat::MIME_TYPE);
echo $result;
//Warning: array_combine(): Both parameters should have an equal number of elements in ///var/www/html/src/Prometheus/RenderTextFormat.php on line 39
//
//Warning: Invalid argument supplied for foreach() in ///var/www/html/src/Prometheus/RenderTextFormat.php on line 40
//
//Warning: Cannot modify header information - headers already sent by (output started at ///var/www/html/src/Prometheus/RenderTextFormat.php:39) in /var/www/html/test.php on line 19
//# HELP test_some_counter it increases
//# TYPE test_some_counter counter
//test_some_counter{} 1

Proposed implementation

Suitable escaping strategy must be used, depending on the storage implementation. JSON-encoding is not enough to escape colons. Base64-encoding the already JSON-encoded values will make the string safe to persist when using APC and in-memory storage.

Can anyone explain how to set a TTL for Counters, Gauges, etc?

prometheus_client_php/src/Prometheus/Storage/Redis.php

Line 187 in 215c9ae

$result = $this->redis->eval(<<<LUA

Cannot create a set of metrices with separate labels.

Hallo,

I am somewhat confused at how to implement multiple metrics with different labels. Consider the following:

http_requests_total{code=200} = 127839
http_requests_total{code=500} = 1234

I cannot see how to implement this in PHP. If given the following code:

<?php 

// This is an example. It's probably  not syntactically correct
$oRegistry = \Prometheus\CollectorRegistry::getDefault();
$iOkRequests = $oRegistry->registerCounter('http', 'requests_total', 'Total number of requests', ['200']);
$iOkRequests->inc(); // This works OK. However, if I want to create an additional metric with another label, such as:

$iBadRequests = $oRegistry->registerCounter('http', 'requests_total', 'Total number of requests', ['code' => 500]); // This fails as the metric has already been registered. See line 111 of CollectorRegistry

I have also tried supplying a 'range of default acceptable values', such as:

<?php 

$oRegistry->registerCounter('http', 'requests_total', 'Total number of requests', ['200', '500']);

// However I cannot increment just the "200" label request as there's an exception checking if the same number of labels are supplied as was earlier indicated. See Collector::assertLabelsAreDefinedCorrectly()

It's quite likely I'm missing something obvious. (:

Cheers! Enjoying playing with the library. See the intended goal at

https://github.com/littlemanco/magento-prometheus

InMemory Functionality not working

Hi, I have tried "in-memory" option

It not working at all. Also its not throwing any error. How to fix this.

Incompatibility with promethus/pushgateway:0.10.0

Hi,

Version 0.10.0 of promethus/pushgateway introduced a change in response:

[CHANGE] Check pushed metrics immediately and reject them if inconsistent. Successful pushes now result in code 200 (not 202)

Current version of this client doesn't support it, throwing an exception (see below)

prometheus_client_php/src/Prometheus/PushGateway.php

Line 85 in 215c9ae

if ($statusCode != 202) {

Add support for Predis

Hello.

I was wondering if there was any appetite of supporting Predis as well as the native Redis extension?

What a useful SDK, but I don't really find how to delete a metric.

Tag new release with new APCu support

Latest release is from 6/Nov, so there is no release with the new features.

switch completely to apcu

I'd like to suggest to solely focus on APCu :

• Starting from PHP7, APCu moved the bc layer into a separate extension (https://pecl.php.net/package/apcu_bc), personally I'd rather avoid installing it.
• APC was mostly relevant before PHP 5.5, those versions all reached EOL and this library explicitly requires PHP >=5.6 anyway.

WDYT?

Create separate APC/APCu adapters.

Would it make sense to create a separate APC and APCu adapter? Currently this package is not usable with PHP 7 since the APC Adapter, which uses APCu methods, is relying on the symfony/polyfill-apcu package under the hood to provide compatibility for PHP 5.6 and APC.

Basically the current APC adapter could be copied and renamed to APCu and instead of apcu_ methods one shall use \apcu_ methods. This way the polyfill could remain.

Invalid metric names should be rejected

$gauge = new Gauge('test', 'some metric', 'this is for testing', array('foo', 'bar'));

The above will output:

# HELP test_some metric "this is for testing".
# TYPE  test_some metric gauge
test_some_metric{foo="abc", bar="def"} 22806528

Note that HELP and TYPE are invalid.
The metric should read test_some_metric. When directly scraping metrics this will cause Prometheus to fail with text format parsing error in line 8: unknown metric type "metric gauge"

Several instances

Hi!
Help us to deal with metrics for several PHP servers. For example we have three servers with 'some_logical_counter' metric. If we use APCu adapter every counter will be incremented independently. So prometheus should pull and sum three 'some_logical_counters'. If we switch to Redis then three instances share the same value.
Does package's storage adapter affect the way prometheus should handle data?

CROSSSLOT Keys in request don't hash to the same slot

When using a Redis cluster that hashes keys to select a node, each key involved in an eval must hash to the same node. Specifying a prefix surrounded by {} should be sufficient to ensure all the keys hash to the same place, but the eval scripts in the Redis adapter inappropriately specify more than the 2 actual keys as KEYS.

Support for Summary collector

Currently gauge, counter and history support is available but summary is missing.

I've already developed this collector and currently running it on some production servers.
It currently only has support for Redis, as we don't use APC and I don't have the time to integrate it.

It would be super-duper if parts of it could be merged or extended.
https://github.com/buffcode/prometheus_client_php/commits/summary-collector

If accepted I will create a PR.

Feature Request: Filesystem adapter

Hallo;

This is a stub to track a PR that I hope to write in future. However, I don't know when, or how, so I'm creating an issue to track interest.

I'd like to implement a Filesystem adapter. There are cases where, in shared hosting, I cannot install additional software onto a machine, or the provisioning of such a software would be terribly complex. However, there is access to a semi permanent directory where I can store files associated with an application, such as in the case of Magento, the var directory. Thus, I would like to implement a filesystem adapter that allows reading current metric state and modification thereof from disk.

Although it might be expensive in terms of IO, I would probably write to disk every time a metric incremented rather than batching the metrics in memory and writing them during destruct. The logic is, in some cases destruct may not be reached during abnormal application shutdown, and metrics would be useful for detecting / diagnosing such cases.

If anyone has any thoughts on this, please comment.

See https://github.com/littlemanco/magento-prometheus for implementation.

Support for timestamp in text format

Hello,

The text exposition format is defined like this

metric_name [
  "{" label_name "=" `"` label_value `"` { "," label_name "=" `"` label_value `"` } [ "," ] "}"
] value [ timestamp ]

I haven't found a way with this library to specify the timestamp of my metrics. It could be great as it allow us to expose past metrics.

Php tests failing right after cloning fresh

I cloned the repository, did "docker-compose up" in one terminal (waited for all images to build and it to start listening), and then ran "docker-compose run phpunit env ADAPTER=redis vendor/bin/phpunit tests/Test/" in another terminal.

However, many tests failed giving
Fatal error: Uncaught exception 'RedisException' with message 'Redis server went away' in /var/www/html/src/Prometheus/Storage/Redis.php:103

When I open another terminal, run redis-cli, followed by PING, I get PONG as a reply suggesting that redis server is running on local host, and thus I do not understand why tests are failing.

Operating system: OSX 10.13.4

Thanks in advance!