Coder Social home page Coder Social logo

prometheus-net / prometheus-net Goto Github PK

View Code? Open in Web Editor NEW
1.8K 31.0 297.0 3.29 MB

.NET library to instrument your code with Prometheus metrics

Home Page: https://prometheus.io

License: MIT License

C# 75.91% PowerShell 0.10% ASP.NET 0.02% HTML 1.50% CSS 0.37% JavaScript 21.87% Dockerfile 0.12% Shell 0.11%
prometheus monitoring metrics net-standard performance-metrics performance-monitor aspnetcore grpc grpc-request-metrics healthchecks

prometheus-net's Introduction

prometheus-net

This is a .NET library for instrumenting your applications and exporting metrics to Prometheus.

Build status Nuget Nuget

The library targets the following runtimes (and newer):

  • .NET Framework 4.6.2
  • .NET 6.0

Table of contents

Best practices and usage

This library allows you to instrument your code with custom metrics and provides some built-in metric collection integrations for ASP.NET Core.

The documentation here is only a minimal quick start. For detailed guidance on using Prometheus in your solutions, refer to the prometheus-users discussion group. You are also expected to be familiar with the Prometheus user guide. /r/PrometheusMonitoring on Reddit may also prove a helpful resource.

Four types of metrics are available: Counter, Gauge, Summary and Histogram. See the documentation on metric types and instrumentation best practices to learn what each is good for.

The Metrics class is the main entry point to the API of this library. The most common practice in C# code is to have a static readonly field for each metric that you wish to export from a given class.

More complex patterns may also be used (e.g. combining with dependency injection). The library is quite tolerant of different usage models - if the API allows it, it will generally work fine and provide satisfactory performance. The library is thread-safe.

Quick start

After installing the library, you should:

  1. Collect some metrics, either by using built-in integrations or publishing your own custom metrics.
  2. Export the collected metrics over an HTTP endpoint (typically /metrics).
  3. Configure a Prometheus server to poll this endpoint for metrics on a regular interval.

Minimal sample app (based on .NET 6 Console app template):

using var server = new Prometheus.KestrelMetricServer(port: 1234);
server.Start();

Console.WriteLine("Open http://localhost:1234/metrics in a web browser.");
Console.WriteLine("Press enter to exit.");
Console.ReadLine();

Refer to the sample projects for quick start instructions:

Name Description
Sample.Web ASP.NET Core application that produces custom metrics and uses multiple integrations to publish built-in metrics
Sample.Console .NET console application that exports custom metrics
Sample.Console.DotNetMeters Demonstrates how to publish custom metrics via the .NET Meters API
Sample.Console.Exemplars .NET console application that attaches exemplars to some metrics
Sample.Console.NetFramework Same as above but targeting .NET Framework
Sample.Console.NoAspNetCore .NET console application that exports custom metrics without requiring the ASP.NET Core runtime to be installed
Sample.Grpc ASP.NET Core application that publishes a gRPC service
Sample.Grpc.Client Client app for the above
Sample.NetStandard Demonstrates how to reference prometheus-net in a .NET Standard class library
Sample.Web.DifferentPort Demonstrates how to set up the metric exporter on a different port from the main web API (e.g. for security purposes)
Sample.Web.MetricExpiration Demonstrates how to use automatic metric deletion
Sample.Web.NetFramework .NET Framework web app that publishes custom metrics

The rest of this document describes how to use individual features of the library.

Installation

Nuget package for general use and metrics export via HttpListener or to Pushgateway: prometheus-net

Install-Package prometheus-net

Nuget package for ASP.NET Core middleware and stand-alone Kestrel metrics server: prometheus-net.AspNetCore

Install-Package prometheus-net.AspNetCore

Nuget package for ASP.NET Core Health Check integration: prometheus-net.AspNetCore.HealthChecks

Install-Package prometheus-net.AspNetCore.HealthChecks

Nuget package for ASP.NET Core gRPC integration: prometheus-net.AspNetCore.Grpc

Install-Package prometheus-net.AspNetCore.Grpc

Nuget package for ASP.NET Web API middleware on .NET Framework: prometheus-net.NetFramework.AspNet

Install-Package prometheus-net.NetFramework.AspNet

Counters

Counters only increase in value and reset to zero when the process restarts.

private static readonly Counter ProcessedJobCount = Metrics
    .CreateCounter("myapp_jobs_processed_total", "Number of processed jobs.");

...

ProcessJob();
ProcessedJobCount.Inc();

Gauges

Gauges can have any numeric value and change arbitrarily.

private static readonly Gauge JobsInQueue = Metrics
    .CreateGauge("myapp_jobs_queued", "Number of jobs waiting for processing in the queue.");

...

jobQueue.Enqueue(job);
JobsInQueue.Inc();

...

var job = jobQueue.Dequeue();
JobsInQueue.Dec();

Histogram

Histograms track the size and number of events in buckets. This allows for aggregatable calculation of quantiles.

private static readonly Histogram OrderValueHistogram = Metrics
    .CreateHistogram("myapp_order_value_usd", "Histogram of received order values (in USD).",
        new HistogramConfiguration
        {
            // We divide measurements in 10 buckets of $100 each, up to $1000.
            Buckets = Histogram.LinearBuckets(start: 100, width: 100, count: 10)
        });

...

OrderValueHistogram.Observe(order.TotalValueUsd);

Summary

Summaries track the trends in events over time (10 minutes by default).

private static readonly Summary RequestSizeSummary = Metrics
    .CreateSummary("myapp_request_size_bytes", "Summary of request sizes (in bytes) over last 10 minutes.");

...

RequestSizeSummary.Observe(request.Length);

By default, only the sum and total count are reported. You may also specify quantiles to measure:

private static readonly Summary RequestSizeSummary = Metrics
    .CreateSummary("myapp_request_size_bytes", "Summary of request sizes (in bytes) over last 10 minutes.",
        new SummaryConfiguration
        {
            Objectives = new[]
            {
                new QuantileEpsilonPair(0.5, 0.05),
                new QuantileEpsilonPair(0.9, 0.05),
                new QuantileEpsilonPair(0.95, 0.01),
                new QuantileEpsilonPair(0.99, 0.005),
            }
        });

The epsilon indicates the absolute error allowed in measurements. For more information, refer to the Prometheus documentation on summaries and histograms.

Measuring operation duration

Timers can be used to report the duration of an operation (in seconds) to a Summary, Histogram, Gauge or Counter. Wrap the operation you want to measure in a using block.

private static readonly Histogram LoginDuration = Metrics
    .CreateHistogram("myapp_login_duration_seconds", "Histogram of login call processing durations.");

...

using (LoginDuration.NewTimer())
{
    IdentityManager.AuthenticateUser(Request.Credentials);
}

Tracking in-progress operations

You can use Gauge.TrackInProgress() to track how many concurrent operations are taking place. Wrap the operation you want to track in a using block.

private static readonly Gauge DocumentImportsInProgress = Metrics
    .CreateGauge("myapp_document_imports_in_progress", "Number of import operations ongoing.");

...

using (DocumentImportsInProgress.TrackInProgress())
{
    DocumentRepository.ImportDocument(path);
}

Counting exceptions

You can use Counter.CountExceptions() to count the number of exceptions that occur while executing some code.

private static readonly Counter FailedDocumentImports = Metrics
    .CreateCounter("myapp_document_imports_failed_total", "Number of import operations that failed.");

...

FailedDocumentImports.CountExceptions(() => DocumentRepository.ImportDocument(path));

You can also filter the exception types to observe:

FailedDocumentImports.CountExceptions(() => DocumentRepository.ImportDocument(path), IsImportRelatedException);

bool IsImportRelatedException(Exception ex)
{
    // Do not count "access denied" exceptions - those are user error for pointing us to a forbidden file.
    if (ex is UnauthorizedAccessException)
        return false;

    return true;
}

Labels

All metrics can have labels, allowing grouping of related time series.

See the best practices on naming and labels.

Taking a counter as an example:

private static readonly Counter RequestCountByMethod = Metrics
    .CreateCounter("myapp_requests_total", "Number of requests received, by HTTP method.", labelNames: new[] { "method" });

...

// You can specify the values for the labels later, once you know the right values (e.g in your request handler code).
RequestCountByMethod.WithLabels("GET").Inc();

NB! Best practices of metric design is to minimize the number of different label values. For example:

  • HTTP request method is a good choice for labeling - there are not many values.
  • URL is a bad choice for labeling - it has many possible values and would lead to significant data processing inefficiency.

Static labels

You can add static labels that always have fixed values. This is possible on two levels:

  • on the metrics registry (e.g. Metrics.DefaultRegistry)
  • on a metric factory (e.g. Metrics.WithLabels())

All levels of labeling can be combined and instance-specific metric labels can also be applied on top, as usual.

Example with static labels on two levels and one instance-specific label:

Metrics.DefaultRegistry.SetStaticLabels(new Dictionary<string, string>
{
  // Labels applied to all metrics in the registry.
  { "environment", "testing" }
});

var backgroundServicesMetricFactory = Metrics.WithLabels(new Dictionary<string, string>
{
  // Labels applied to all metrics created via this factory.
  { "category", "background-services" }
});

var requestsHandled = backgroundServicesMetricFactory
  .CreateCounter("myapp_requests_handled_total", "Count of requests handled, labelled by response code.", labelNames: new[] { "response_code" });

// Labels applied to individual instances of the metric.
requestsHandled.WithLabels("404").Inc();
requestsHandled.WithLabels("200").Inc();

Exemplars

Exemplars facilitate distributed tracing, by attaching related trace IDs to metrics. This enables a metrics visualization app to cross-reference traces that explain how the metric got the value it has.

See also, Grafana fundamentals - introduction to exemplars.

By default, prometheus-net will create an exemplar with the trace_id and span_id labels based on the current distributed tracing context (Activity.Current). If using OpenTelemetry tracing with ASP.NET Core, the traceparent HTTP request header will be used to automatically assign Activity.Current.

private static readonly Counter TotalSleepTime = Metrics
    .CreateCounter("sample_sleep_seconds_total", "Total amount of time spent sleeping.");
...

// You only need to create the Activity if one is not automatically assigned (e.g. by ASP.NET Core).
using (var activity = new Activity("Pausing before record processing").Start())
{
    var sleepStopwatch = Stopwatch.StartNew();
    await Task.Delay(TimeSpan.FromSeconds(1));

    // The trace_id and span_id from the current Activity are exposed as the exemplar.
    TotalSleepTime.Inc(sleepStopwatch.Elapsed.TotalSeconds);
}

This will be published as the following metric point:

sample_sleep_seconds_total 251.03833569999986 # {trace_id="08ad1c8cec52bf5284538abae7e6d26a",span_id="4761a4918922879b"} 1.0010688 1672634812.125

You can override any default exemplar logic by providing your own exemplar when updating the value of the metric:

private static readonly Counter RecordsProcessed = Metrics
    .CreateCounter("sample_records_processed_total", "Total number of records processed.");

// The key from an exemplar key-value pair should be created once and reused to minimize memory allocations.
private static readonly Exemplar.LabelKey RecordIdKey = Exemplar.Key("record_id");
...

foreach (var record in recordsToProcess)
{
    var exemplar = Exemplar.From(RecordIdKey.WithValue(record.Id.ToString()));
    RecordsProcessed.Inc(exemplar);
}

Warning Exemplars are limited to 128 ASCII characters (counting both keys and values) - they are meant to contain IDs for cross-referencing with trace databases, not as a replacement for trace databases.

Exemplars are only published if the metrics are being scraped by an OpenMetrics-capable client. For development purposes, you can force the library to use the OpenMetrics exposition format by adding ?accept=application/openmetrics-text to the /metrics URL.

Note The Prometheus database automatically negotiates OpenMetrics support when scraping metrics - you do not need to apply any special scraping configuration in production scenarios. You may need to enable exemplar storage, though.

See also, Sample.Console.Exemplars.

Limiting exemplar volume

Exemplars can be expensive to store in the metrics database. For this reason, it can be useful to only record exemplars for "interesting" metric values.

You can use ExemplarBehavior.NewExemplarMinInterval to define a minimum interval between exemplars - a new exemplar will only be recorded if this much time has passed. This can be useful to limit the rate of publishing unique exemplars.

You can customize the default exemplar provider via IMetricFactory.ExemplarBehavior or CounterConfiguration.ExemplarBehavior and HistogramConfiguration.ExemplarBehavior, which allows you to provide your own method to generate exemplars and to filter which values/metrics exemplars are recorded for:

Example of a custom exemplar provider used together with exemplar rate limiting:

// For the next histogram we only want to record exemplars for values larger than 0.1 (i.e. when record processing goes slowly).
static Exemplar RecordExemplarForSlowRecordProcessingDuration(Collector metric, double value)
{
    if (value < 0.1)
        return Exemplar.None;

    return Exemplar.FromTraceContext();
}

var recordProcessingDuration = Metrics
    .CreateHistogram("sample_record_processing_duration_seconds", "How long it took to process a record, in seconds.",
    new HistogramConfiguration
    {
        Buckets = Histogram.PowersOfTenDividedBuckets(-4, 1, 5),
        ExemplarBehavior = new()
        {
            DefaultExemplarProvider = RecordExemplarForSlowRecordProcessingDuration,
            // Even if we have interesting data more often, do not record it to conserve exemplar storage.
            NewExemplarMinInterval = TimeSpan.FromMinutes(5)
        }
    });

For the ASP.NET Core HTTP server metrics, you can further fine-tune exemplar recording by inspecting the HTTP request and response:

app.UseHttpMetrics(options =>
{
    options.ConfigureMeasurements(measurementOptions =>
    {
        // Only measure exemplar if the HTTP response status code is not "OK".
        measurementOptions.ExemplarPredicate = context => context.Response.StatusCode != HttpStatusCode.Ok;
    });
});

When are metrics published?

Metrics without labels are published immediately after the Metrics.CreateX() call. Metrics that use labels are published when you provide the label values for the first time.

Sometimes you want to delay publishing a metric until you have loaded some data and have a meaningful value to supply for it. The API allows you to suppress publishing of the initial value until you decide the time is right.

private static readonly Gauge UsersLoggedIn = Metrics
    .CreateGauge("myapp_users_logged_in", "Number of active user sessions",
        new GaugeConfiguration
        {
            SuppressInitialValue = true
        });

...

// After setting the value for the first time, the metric becomes published.
UsersLoggedIn.Set(LoadSessions().Count);

You can also use .Publish() on a metric to mark it as ready to be published without modifying the initial value (e.g. to publish a zero). Conversely, you can use .Unpublish() to hide a metric temporarily. Note that the metric remains in memory and retains its value.

Deleting metrics

You can use .Dispose() or .RemoveLabelled() methods on the metric classes to manually delete metrics at any time.

In some situations, it can be hard to determine when a metric with a specific set of labels becomes irrelevant and needs to be removed. The library provides some assistance here by enabling automatic expiration of metrics when they are no longer used.

To enable automatic expiration, create the metrics via the metric factory returned by Metrics.WithManagedLifetime(). All such metrics will have a fixed expiration time, with the expiration restarting based on certain conditions that indicate the metric is in use.

Option 1: metric lifetime can be controlled by leases - the metric expiration timer starts when the last lease is released (and will be reset when a new lease is taken again).

var factory = Metrics.WithManagedLifetime(expiresAfter: TimeSpan.FromMinutes(5));

// With expiring metrics, we get back handles to the metric, not the metric directly.
var inProgressHandle = expiringMetricFactory
  .CreateGauge("documents_in_progress", "Number of documents currently being processed.",
    // Automatic metric deletion only makes sense if we have a high/unknown cardinality label set,
    // so here is a sample label for each "document provider", whoever that may be.
    labelNames: new[] { "document_provider" });

...

public void ProcessDocument(string documentProvider)
{
  // Automatic metric deletion will not occur while this lease is held.
  // This will also reset any existing expiration timer for this document provider.
  inProgressHandle.WithLease(metric =>
  {
    using (metric.TrackInProgress())
      DoDocumentProcessingWork();
  }, documentProvider);
  // Lease is released here.
  // If this was the last lease for this document provider, the expiration timer will now start.
}

Scenario 2: sometimes managing the leases is not required because you simply want the metric lifetime to be extended whenever the value is updated.

var factory = Metrics.WithManagedLifetime(expiresAfter: TimeSpan.FromMinutes(5));

// With expiring metrics, we get back handles to the metric, not the metric directly.
var processingStartedHandle = expiringMetricFactory
  .CreateGauge("documents_started_processing_total", "Number of documents for which processing has started.",
    // Automatic metric deletion only makes sense if we have a high/unknown cardinality label set,
    // so here is a sample label for each "document provider", whoever that may be.
    labelNames: new[] { "document_provider" });

// This returns a metric instance that will reset the expiration timer whenever the metric value is updated.
var processingStarted = processingStartedHandle.WithExtendLifetimeOnUse();

...

public void ProcessDocument(string documentProvider)
{
  // This will reset the expiration timer for this document provider.
  processingStarted.WithLabels(documentProvider).Inc();

  DoDocumentProcessingWork();
}

The expiration logic is scoped to the factory. Multiple handles for the same metric from the same factory will share their expiration logic. However, handles for the same metric from different factories will have independent expiration logic.

See also, Sample.Web.MetricExpiration.

ASP.NET Core exporter middleware

For projects built with ASP.NET Core, a middleware plugin is provided.

If you use the default Visual Studio project templates, modify the UseEndpoints call as follows:

  • Add endpoints.MapMetrics() anywhere in the delegate body.
public void Configure(IApplicationBuilder app, ...)
{
    // ...

    app.UseEndpoints(endpoints =>
    {
        // ...

        endpoints.MapMetrics();
    });
}

The default configuration will publish metrics on the /metrics URL.

The ASP.NET Core functionality is delivered in the prometheus-net.AspNetCore NuGet package.

See also, Sample.Web.

ASP.NET Core HTTP request metrics

The library exposes some metrics from ASP.NET Core applications:

  • Number of HTTP requests in progress.
  • Total number of received HTTP requests.
  • Duration of HTTP requests.

The ASP.NET Core functionality is delivered in the prometheus-net.AspNetCore NuGet package.

You can expose HTTP metrics by modifying your Startup.Configure() method:

  • After app.UseRouting() add app.UseHttpMetrics().

Example Startup.cs:

public void Configure(IApplicationBuilder app, ...)
{
    // ...

    app.UseRouting();
    app.UseHttpMetrics();

    // ...
}

By default, metrics are collected separately for each response status code (200, 201, 202, 203, ...). You can considerably reduce the size of the data set by only preserving information about the first digit of the status code:

app.UseHttpMetrics(options =>
{
    // This will preserve only the first digit of the status code.
    // For example: 200, 201, 203 -> 2xx
    options.ReduceStatusCodeCardinality();
});

NB! Exception handler middleware that changes HTTP response codes must be registered after UseHttpMetrics() in order to ensure that prometheus-net reports the correct HTTP response status code.

The action, controller and endpoint route parameters are always captured by default. If Razor Pages is in use, the page label will be captured to show the path to the page.

You can include additional route parameters as follows:

app.UseHttpMetrics(options =>
{
    // Assume there exists a custom route parameter with this name.
    options.AddRouteParameter("api-version");
});

You can also extract arbitrary data from the HttpContext into label values as follows:

app.UseHttpMetrics(options =>
{
    options.AddCustomLabel("host", context => context.Request.Host.Host);
});

See also, Sample.Web.

ASP.NET Core gRPC request metrics

The library allows you to expose some metrics from ASP.NET Core gRPC services. These metrics include labels for service and method name.

You can expose gRPC metrics by modifying your Startup.Configure() method:

  • After app.UseRouting() add app.UseGrpcMetrics().

Example Startup.cs:

public void Configure(IApplicationBuilder app, ...)
{
    // ...

    app.UseRouting();
    app.UseGrpcMetrics();

    // ...
}

The gRPC functionality is delivered in the prometheus-net.AspNetCore.Grpc NuGet package.

See also, Sample.Grpc.

IHttpClientFactory metrics

This library allows you to expose metrics about HttpClient instances created using IHttpClientFactory.

The exposed metrics include:

  • Number of HTTP requests in progress.
  • Total number of started HTTP requests.
  • Duration of HTTP client requests (from start of request to end of reading response headers).
  • Duration of HTTP client responses (from start of request to end of reading response body).

Example Startup.cs modification to enable these metrics for all HttpClients registered in the service collection:

public void ConfigureServices(IServiceCollection services)
{
    // ...

    services.UseHttpClientMetrics();

    // ...
}

Note You can also register HTTP client metrics only for a specific HttpClient by calling services.AddHttpClient(...).UseHttpClientMetrics().

See also, Sample.Web.

ASP.NET Core health check status metrics

You can expose the current status of ASP.NET Core health checks as Prometheus metrics by extending your IHealthChecksBuilder in the Startup.ConfigureServices() method:

public void ConfigureServices(IServiceCollection services, ...)
{
    // ...

    services.AddHealthChecks()
        // ...
        <Your Health Checks>
        // ...
        .ForwardToPrometheus();

    // ...
}

The status of each health check will be published in the aspnetcore_healthcheck_status metric.

The ASP.NET Core health check integration is delivered in the prometheus-net.AspNetCore.HealthChecks NuGet package.

See also, Sample.Web.

Protecting the metrics endpoint from unauthorized access

You may wish to restrict access to the metrics export URL. Documentation on how to apply ASP.NET Core security mechanisms is beyond the scope of this readme file but a good starting point may be to require an authorization policy to be satisfied for accessing the endpoint

app.UseEndpoints(endpoints =>
{
    // ...

    // Assumes that you have previously configured the "ReadMetrics" policy (not shown).
    endpoints.MapMetrics().RequireAuthorization("ReadMetrics");
});

Another commonly used option is to expose a separate web server endpoint (e.g. a new KestrelMetricServer instance) on a different port, with firewall rules limiting access to only certain IP addresses. Refer to the sample project Sample.Web.DifferentPort.

ASP.NET Web API exporter

The easiest way to export metrics from an ASP.NET Web API app on the full .NET Framework is to use AspNetMetricServer in your Global.asax.cs file. Insert the following line to the top of the Application_Start method:

protected void Application_Start(object sender, EventArgs e)
{
    AspNetMetricServer.RegisterRoutes(GlobalConfiguration.Configuration);

    // Other code follows.
}

The above snippet exposes metrics on the /metrics URL.

The AspNetMetricServer class is provided by the prometheus-net.NetFramework.AspNet NuGet package.

Kestrel stand-alone server

In some situation, you may wish to start a stand-alone metric server using Kestrel (e.g. if your app has no other HTTP-accessible functionality).

var metricServer = new KestrelMetricServer(port: 1234);
metricServer.Start();

The default configuration will publish metrics on the /metrics URL.

If your app is an ASP.NET Core web app, you can use a pipeline-integrated mechanism:

services.AddMetricServer(options =>
{
    options.Port = 1234;
});

Publishing to Pushgateway

Metrics can be posted to a Pushgateway server.

var pusher = new MetricPusher(new MetricPusherOptions
{
    Endpoint = "https://pushgateway.example.org:9091/metrics",
    Job = "some_job"
});

pusher.Start();

Note that the default behavior of the metric pusher is to append metrics. You can use MetricPusherOptions.ReplaceOnPush to make it replace existing metrics in the same group, removing any that are no longer pushed.

Publishing to Pushgateway with basic authentication

You can use a custom HttpClient to supply credentials for the Pushgateway.

// Placeholder username and password here - replace with your own data.
var headerValue = Convert.ToBase64String(Encoding.UTF8.GetBytes("username:password"));
var httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", headerValue);

var pusher = new MetricPusher(new MetricPusherOptions
{
    Endpoint =  "https://pushgateway.example.org:9091/metrics",
    Job = "some_job",
    HttpClientProvider = () => httpClient
});

pusher.Start();

Publishing via standalone HTTP handler

As a fallback option for scenarios where Kestrel or ASP.NET Core hosting is unsuitable, an HttpListener based metrics server implementation is also available.

var metricServer = new MetricServer(port: 1234);
metricServer.Start();

The default configuration will publish metrics on the /metrics URL.

MetricServer.Start() may throw an access denied exception on Windows if your user does not have the right to open a web server on the specified port. You can use the netsh command to grant yourself the required permissions:

netsh http add urlacl url=http://+:1234/metrics user=DOMAIN\user

Publishing raw metrics document

In scenarios where you handle publishing via a custom endpoint, you can export the entire metrics data set as a Prometheus text document.

await Metrics.DefaultRegistry.CollectAndExportAsTextAsync(outputStream);

Just-in-time updates

In some scenarios you may want to only collect data when it is requested by Prometheus. To easily implement this scenario prometheus-net enables you to register a callback before every collection occurs. Register your callback using Metrics.DefaultRegistry.AddBeforeCollectCallback().

Every callback will be executed before each collection, which will not finish until every callback has finished executing. Prometheus will expect each scrape to complete within a certain amount of seconds. To avoid timeouts, ensure that any registered callbacks execute quickly.

  • A synchronous callback (of type Action) should not take more than a few milliseconds. Do not read data from remote systems in these callbacks.
  • An asynchronous callback (of type Func<CancellationToken, Task>) is more suitable for long-running data collection work (lasting a few seconds). You can use asynchronous callbacks for reading data from remote systems.
Metrics.DefaultRegistry.AddBeforeCollectCallback(async (cancel) =>
{
    // Probe a remote system.
    var response = await httpClient.GetAsync("https://google.com", cancel);

    // Increase a counter by however many bytes we loaded.
    googlePageBytes.Inc(response.Content.Headers.ContentLength ?? 0);
});

Suppressing default metrics

The library enables various default metrics and integrations by default. If these default metrics are not desirable you may remove them by calling Metrics.SuppressDefaultMetrics() before registering any of your own metrics.

DiagnosticSource integration

.NET Core provides the DiagnosticSource mechanism for reporting diagnostic events, used widely by .NET and ASP.NET Core classes. To expose basic data on these events via Prometheus, you can use the DiagnosticSourceAdapter class:

// An optional "options" parameter is available to customize adapter behavior.
var registration = DiagnosticSourceAdapter.StartListening();

...

// Stops listening for DiagnosticSource events.
registration.Dispose();

Any events that occur are exported as Prometheus metrics, indicating the name of the event source and the name of the event:

diagnostic_events_total{source="Microsoft.AspNetCore",event="Microsoft.AspNetCore.Mvc.AfterAction"} 4
diagnostic_events_total{source="HttpHandlerDiagnosticListener",event="System.Net.Http.Request"} 8

The level of detail obtained from this is rather low - only the total count for each event is exported. For more fine-grained analytics, you need to listen to DiagnosticSource events on your own and create custom metrics that can understand the meaning of each particular type of event that is of interest to you.

EventCounter integration

Note The output produced by this integration has changed significantly between prometheus-net 6.0 and prometheus-net 7.0. The old output format is no longer supported.

.NET Core provides the EventCounter mechanism for reporting diagnostic events, used used widely by .NET and ASP.NET Core classes. This library publishes all .NET EventCounter data by default. To suppress this, see Suppressing default metrics.

You can configure the integration using Metrics.ConfigureEventCounterAdapter().

By default, prometheus-net will only publish the well-known .NET EventCounters to minimize resource consumption in the default configuration. A custom event source filter must be provided in the configuration to enable publishing of additional event counters.

See also, Sample.Console.

.NET Meters integration

Note The output produced by this integration has changed significantly between prometheus-net 6.0 and prometheus-net 7.0. The old output format is no longer supported.

.NET provides the Meters mechanism for reporting diagnostic metrics. This library publishes all .NET Meters API data by default. To suppress this, see Suppressing default metrics.

You can configure the integration using Metrics.ConfigureMeterAdapter().

See also, Sample.Console.DotNetMeters.

Benchmarks

A suite of benchmarks is included if you wish to explore the performance characteristics of the library. Simply build and run the Benchmarks.NetCore project in Release mode.

As an example of the performance of measuring data using prometheus-net, we have the results of the MeasurementBenchmarks here, converted into measurements per second:

Metric type Measurements per second
Counter 261 million
Gauge 591 million
Histogram (16 buckets) 105 million
Histogram (128 buckets) 65 million

Another popular .NET SDK with Prometheus support is the OpenTelemetry SDK. To help you choose, we have SdkComparisonBenchmarks.cs to compare the two SDKs and give some idea of how they differer in the performance tradeoffs made. Both SDKs are evaluated in single-threaded mode under a comparable workload and enabled feature set. A representative result is here:

SDK Benchmark scenario CPU time Memory
prometheus-net Counter (existing timeseries) x100K 230 ยตs None
OpenTelemetry Counter (existing timeseries) x100K 10998 ยตs None
prometheus-net Histogram (existing timeseries) x100K 957 ยตs None
OpenTelemetry Histogram (existing timeseries) x100K 12110 ยตs None
prometheus-net Histogram (new timeseries) x1K 716 ยตs 664 KB
OpenTelemetry Histogram (new timeseries) x1K 350 ยตs 96 KB

Community projects

Some useful related projects are:

Note: to avoid confusion between "official" prometheus-net and community maintained packages, the prometheus-net namespace is protected on nuget.org. However, the prometheus-net.Contrib.* namespace allows package publishing by all authors.

prometheus-net's People

Contributors

achehre avatar andrasm avatar avendel avatar chuseman avatar cmannix avatar daniel15 avatar dmirmilshteyn avatar hsyed-dojo avatar kenkam avatar lakario avatar macaba avatar marcel-b avatar marcowesterink avatar mattias01 avatar medved91 avatar mee7891 avatar mihamarkic avatar pamir avatar qed- avatar rgl avatar rocklan avatar runerys avatar rwkarg avatar sandersaares avatar sderen avatar steven-dawkins avatar utpilla avatar valentynpopov avatar weihanli avatar zmey-gorynych avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

prometheus-net's Issues

Allow setting counter value when exposing existing counts

I am just started to use prometheus, and your library and have come across this issue.

Counter only allows you to increase the value which is fine if you actually using it to count things but breaks down if you have an existing counter you are trying to expose though prometheus.

E.g. the DotNetStatsCollector seems to increment based on a calculated diff.
_cpuTotal.Inc(_process.TotalProcessorTime.TotalSeconds - _cpuTotal.Value);
This looks like a race condition to me as I cannot see anything stopping two scrapes occurring at the same time. In fact as I understand it multiple parallel scrapes is the preferred way to achive high availability of the prometheus server.

MetricServer.Stop still cause "ObjectDisposedException"

Hello.

I am still experiencing "ObjectDisposedException" on server stop.
Here is exception detail:

Cannot access a disposed object.
Object name: System.Net.HttpListener.

at System.Net.HttpListener.CheckDisposed()
at System.Net.HttpListener.BeginGetContext(AsyncCallback callback, Object state)
at Prometheus.MetricServer.b__6_0(Action repeatAction)
at System.Reactive.Concurrency.Scheduler.b__45(Action1 _action, Action1 self)
at System.Reactive.Concurrency.Scheduler.<>c__DisplayClass501.<InvokeRec1>b__4d(TState state1) at System.Reactive.Concurrency.Scheduler.<>c__DisplayClass501.<>c__DisplayClass52.b__4f(IScheduler scheduler1, TState state3)
at System.Reactive.Concurrency.DefaultScheduler.<>c__DisplayClass1`1.b__0(Object _)
at System.Reactive.Concurrency.DefaultConcurrencyAbstractionLayer.<>c__DisplayClass1.b__0(Object _)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

How to ensure that unlabelled metrics are created when appropriate

By default, unlabelled metrics are only created if you give them some value. However, it can be that sometimes you have metrics that rarely deviate from zero values. How to ensure that they are still created even if zero? While not creating them when they are truly not desired.

Is this just about documentation? How about always creating them if no label names are specified for the metric? That seems the most straightforward course.

Document how to suppress the default .NET process metrics

They are not needed in some scenarios. Really, they are just an easy way to test the library and unlikely to be very beneficial in the first place. There should be a section in the readme that says how to easily disable them.

Allow removal of specific labelled series without removing collector

Sometimes the data I am monitoring changes and a labelled series becomes meaningless - there are no more values to report for it. However, there does not appear to be any way to actually remove such data at present - I can only delete an entire collector, not an individual labelled series within.

It would be desirable to have such a feature.

Built-in HTTP server metrics middleware

any plans on adding http_server metrics on the middleware?

talking about duration of requests or total count of requests, like the ruby or go client does.

Add query parameters to Collector interface

It would be nice to be able to receive query parameters in a custom collector. This way I can send arguments from prometheus to my custom built exporter (like it's done in the official blackbox_exporter.

http://my-custom-exporter/metrics?target=http:google.ch

`RegisterMetrics()` in `IOnDemandCollector` should provide a `MetricFactory` parameter

I'm currently creating several exporters, most of which utilize IOnDemandCollector style metrics. This means that there are several steps that I have to go through to be successful:

  1. I must create an instance of ICollectorRegistry (and the only provided implementation is the confusingly-named DefaultCollectorRegistry).
  2. I must instantiate my IOnDemandCollector instance(s), providing my ICollectorRegistry instance to the constructor.
  3. I must cast the ICollectorRegistry to a DefaultCollectorRegistry in order to call the RegisterOnDemandCollectors method.
  4. Inside my RegisterMetrics() method, I must use the previously-passed ICollectorRegistry instance to call Metrics.WithCustomRegistry() with.

This sequence would be somewhat simpler and less error-prone if either the ICollectorRegistry instance or a MetricFactory instance was passed into the RegisterMetrics() method.

How to apply Basic Auth for /metrics route?

Hello,

Right now I can expose /metrics route which will be pulled by Prometheus. Does this library support Basic Auth for the /metrics route?. To prevent unauthorized access to /metrics.

Thank you for this great library :)

Idea: stop the collector from throwing an exception if the label format is not permitted

Rather than doing that, which might happen at runtime rather than during unit testing, maybe it could clean the invalid characters out of the label names instead and maybe log a warning?

For example:

public class LabelCleaner
    {
        readonly static Regex LabelNameRegex = new Regex("[^a-zA-Z_:0-9]");

        public static string[] Clean(IEnumerable<string> labels)
        {
            return labels.Select(l => clean(l)).ToArray();
        }

        private static string clean(string label)
        {
            if (char.IsDigit(label.FirstOrDefault()))
            {
                return clean("l_" + label);
            }
            return LabelNameRegex.Replace(label, "_");
        }
    }
public class LabelCleanerTests
    {
        [Fact]
        public void SpacesAreReplaced()
        {
            Run(" ", "_");
        }

        [Fact]
        public void AmpersandsAreReplaced()
        {
            Run("&", "_");
        }

        [Fact]
        public void BracketsAreReplaced()
        {
            Run("{}", "__");
        }

        [Fact]
        public void ValidCharactersAreNotReplaced()
        {
            Run("this_is_valid", "this_is_valid");
        }

        [Fact]
        public void MustStartWithALetter()
        {
            Run("123_not_valid", "l_123_not_valid");
        }

        [Fact]
        public void EmptyLabelsDoNotThrowExceptions()
        {
            Run("", "");
        }

        public void Run(string input, string expected)
        {
            var result = LabelCleaner.Clean(new string[] { input });
            Assert.Equal(expected, result[0]);
        }
    }

Prometheus.Tests.ThreadSafeDoubleTests.ThreadSafeDouble_Overrides fails

I am not familiar enough to be able to tell if this is broken test logic or broken code but it sounds like string formatting is wrong somewhere.

1) Failed : Prometheus.Tests.ThreadSafeDoubleTests.ThreadSafeDouble_Overrides
  String lengths are both 4. Strings differ at index 1.
  Expected: "9.15"
  But was:  "9,15"
  ------------^
at Prometheus.Tests.ThreadSafeDoubleTests.ThreadSafeDouble_Overrides() in c:\Source\prometheus-net\prometheus-net.sharedtests\ThreadSafeDoubleTests.cs:line 34

Counter allows no way to reset to zero

Currently, the Counter class provides as its only way to alter the value the Inc() method. This method only takes values >= 0, which means that there is no way to reset the Counter and no way to decrement its value.

I am using the Counter class to mirror a value from a Windows performance counter, which acts as a counter. Because I have no way to interact with the Counter other than increment it, I am required to track each value that I retrieve from the performance counter, and compare it against each new sample, and do the math myself to determine how much it's incremented.

It's my understanding that the Prometheus server itself does this math. The documentation at https://prometheus.io/docs/instrumenting/writing_clientlibs/ allows client libraries to offer a method to reset Counter values to zero. I believe this would be a useful functionality, but it is not enough alone to eliminate the requirement for client code to maintain the previously sampled value and compare the current value.

Ideas for 2.x

A few ideas for version 2 of the library.

Consider splitting into a core library and separate server libraries.

Right now the single package includes both the core metrics collection registry, scrape handler, and server implementations. The server implementations carry a lot of baggage which aren't always needed (System.Reactive, Microsoft.AspNetCore.Server, etc).

Consider a large project which may want to instrument some internal library; it would be convenient include a reference to a small core prometheus-net package that doesn't bring any other dependencies. But the "root" project, or projects, would then reference the appropriate hosting packages.

My proposal:

  • Prometheus.Core (has no dependencies outside of .NET)
  • Prometheus.Hosting.Core (ScrapeHandler and formatters)
  • Prometheus.Hosting.HttpListener
  • Prometheus.Hosting.Kestral

webapi usage example missing

wondering if I missing something, but it is really not clear from readme how to get started in good old .net webapi, e.g. what should we add and where to Global.asax, or may be Startup.cs or ...

netstandard13 support

Is this impractical now? If I were to multitarget this into a pull request, would you consider pulling it in?

Unlabelled histogram is always created by default

When creating histograms with labels, the unlabeled instance is always reported even if you never observe any value without labels. The culprit is following line in Histogram's constructor:

            Unlabelled.Init(this, LabelValues.Empty);

Installing with .net 4.6.1

Hi guys,

I am using older operationg system Windows Server 2008 R2 with installed .Net 4.6.1, VS 2015 Update 3, and I have run into following issue:

Attempting to gather dependency information for package 'prometheus-net.2.1.2' with respect to project 'ConsoleApplication1', targeting '.NETFramework,Version=v4.6.1'
Attempting to resolve dependencies for package 'prometheus-net.2.1.2' with DependencyBehavior 'Lowest'
Resolving actions to install package 'prometheus-net.2.1.2'
Resolved actions to install package 'prometheus-net.2.1.2'
GET https://api.nuget.org/v3-flatcontainer/protobuf-net/2.3.4/protobuf-net.2.3.4.nupkg
OK https://api.nuget.org/v3-flatcontainer/protobuf-net/2.3.4/protobuf-net.2.3.4.nupkg 575ms
Installing protobuf-net 2.3.4.
Adding package 'protobuf-net.2.3.4' to folder 'c:\users\ha#mag\documents\visual studio 2015\Projects\ConsoleApplication1\packages'
Added package 'protobuf-net.2.3.4' to folder 'c:\users\ha#mag\documents\visual studio 2015\Projects\ConsoleApplication1\packages'
Added package 'protobuf-net.2.3.4' to 'packages.config'
Successfully installed 'protobuf-net 2.3.4' to ConsoleApplication1
GET https://api.nuget.org/v3-flatcontainer/prometheus-net/2.1.2/prometheus-net.2.1.2.nupkg
OK https://api.nuget.org/v3-flatcontainer/prometheus-net/2.1.2/prometheus-net.2.1.2.nupkg 581ms
Installing prometheus-net 2.1.2.
Install failed. Rolling back...
Package 'prometheus-net.2.1.2 : protobuf-net [2.3.4, )' does not exist in project 'ConsoleApplication1'
Removed package 'protobuf-net.2.3.4' from 'packages.config'
Package 'prometheus-net.2.1.2 : protobuf-net [2.3.4, )' does not exist in folder 'c:\users\ha#mag\documents\visual studio 2015\Projects\ConsoleApplication1\packages'
Removing package 'protobuf-net.2.3.4' from folder 'c:\users\ha#mag\documents\visual studio 2015\Projects\ConsoleApplication1\packages'
Removed package 'protobuf-net.2.3.4' from folder 'c:\users\ha#mag\documents\visual studio 2015\Projects\ConsoleApplication1\packages'
Could not install package 'prometheus-net 2.1.2'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.6.1', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.
========== Finished ==========

I hope you can help me with that.

Thank you in advance,

Martin Bodocky

Building solution and running "tester" results in exception

I cloned the solution and ran the included "tester" utility but it apparently crashes due to missing protobuf-net assembly binding redirect.

Unhandled Exception: System.TypeInitializationException: The type initializer for 'Prometheus.Advanced.Collector`1' threw an exception. ---> System.TypeInitializationException: The type initializer for 'Prometheus.Internal.LabelValues' threw an exception. ---> System.IO.FileLoadException: Could not load file or assembly 'protobuf-net, Version=2.0.0.668, Culture=neutral, PublicKeyToken=257b51d87d2e4d67' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at Prometheus.Internal.LabelValues..ctor(String[] names, String[] values)
   at Prometheus.Internal.LabelValues..cctor() in c:\Source\prometheus-net\prometheus-net.shared\Internal\LabelValues.cs:line 12
   --- End of inner exception stack trace ---
   at Prometheus.Internal.LabelValues..ctor(String[] names, String[] values)
   at Prometheus.Advanced.Collector`1..cctor() in c:\Source\prometheus-net\prometheus-net.shared\Advanced\Collector.cs:line 22
   --- End of inner exception stack trace ---
   at Prometheus.Advanced.Collector`1..ctor(String name, String help, String[] labelNames) in c:\Source\prometheus-net\prometheus-net.shared\Advanced\Collector.cs:line 54
   at Prometheus.Counter..ctor(String name, String help, String[] labelNames) in c:\Source\prometheus-net\prometheus-net.shared\Counter.cs:line 18
   at Prometheus.Advanced.MetricFactory.CreateCounter(String name, String help, String[] labelNames) in c:\Source\prometheus-net\prometheus-net.shared\Advanced\MetricFactory.cs:line 18
   at Prometheus.Metrics.CreateCounter(String name, String help, String[] labelNames) in c:\Source\prometheus-net\prometheus-net.shared\Metrics.cs:line 14
   at Prometheus.Advanced.DotNetStatsCollector.RegisterMetrics() in c:\Source\prometheus-net\prometheus-net.shared\Advanced\DotNetStatsCollector.cs:line 32
   at Prometheus.Advanced.DefaultCollectorRegistry.RegisterOnDemandCollectors(IEnumerable`1 onDemandCollectors) in c:\Source\prometheus-net\prometheus-net.shared\Advanced\DefaultCollectorRegistry.cs:line 21
   at Prometheus.MetricHandler..ctor(IEnumerable`1 standardCollectors, ICollectorRegistry registry) in c:\Source\prometheus-net\prometheus-net.shared\MetricHandler.cs:line 26
   at Prometheus.MetricServer..ctor(String hostname, Int32 port, IEnumerable`1 standardCollectors, String url, ICollectorRegistry registry, Boolean useHttps) in c:\Source\prometheus-net\prometheus-net.shared\MetricServer.cs:line 18
   at tester.MetricServerTester.InitializeMetricHandler() in c:\Source\prometheus-net\tester\MetricServerTester.cs:line 16
   at tester.Program.Main(String[] args) in c:\Source\prometheus-net\tester\Program.cs:line 17

Suggestion: Add documentation on Collectors

First off, thanks for creating this library. I've been using it for a couple months now and I think that it's laid out well, and it does precisely what I need it to do.

One thing that confused me, though, starting out was how to hook into things so that I could periodically set the current state of some of my metrics (like current number of messages, RAM usage, etc.). I started creating a class with a Timer that would periodically update those sorts of metrics... and then later realized that I just needed to use your Collector framework, and that you already created a Collector by default when starting up the server that provided the exact items that I was trying to log.

I'd like to suggest adding some documentation on Collectors, and perhaps moving them from the Advanced folder to a Collectors folder or something similar. As I was new to Prometheus, I was kind of trying to shy away in general from more advanced stuff, but I didn't think that something like what I wanted (a collector) was really an advanced feature, so I didn't go looking for it there either. My fault, probably, but still, I think we could make it clearer so that people don't go trying to reinvent what you've already put in.

If you accept pull requests, I'd be happy to create the documentation and get a pull request in to you.

Thanks again!

ScrapeHandler is internal (instead of public), preventing usage without MetricServer

I'm trying to use prometheus-net in an ASP.NET MVC app and want to expose metrics directly in one of my controllers instead of using MetricServer. It seems like a combination of Registry and ScrapeHandler would do the job, but ScrapeHandler is internal so we can't use it externally.

Unless I've misunderstood how external usage is supposed to work, it looks like this was simply forgotten in commit 948e9f6.

Any help would be appreciated!

Document how to expose external metrics as Prometheus metrics

As shown in #32 and #63, there is a quite relevant use case where external (non-Prometheus) sources for metrics are queried and the results exported for Prometheus. Examples are Windows performance counters or arbitrary APIs.

Doing this by simply mirroring the external metrics using prometheus-net builtin collectors can get a bit hairy, as for example counters are not supposed to ever decrease, so you have to keep track of the external source's changes instead of the current value in order to expose the value decorated as a counter.

While this may be workable in many cases, prometheus-net also supports the creation of custom collectors that do not have the limitations of the builtin collectors. This feature is insufficiently discoverable and should be better documented.

Prometheus documentation suggests that for exposing external metrics, one should return a fresh data set on every collection, discarding it immediately thereafter. While there are downsides to this (e.g. external systems might be too slow to respond with their data) it is possible to implement this with prometheus-net by creating a custom collector registry. This feature is insufficiently discoverable and should be better documented.

Change metric type names to lowercase

After upgrading to Prometheus v2.4.0, the metrics endpoint exposed by my .NET services stopped working with the following error: invalid metric type "HISTOGRAM"

image

A related issue has been opened in the Prometheus repo, but apparently the fix should be done in the client library, as it's not reporting the metric names as documented here.

Ref: prometheus/prometheus#4602

Edit. More information:

  • Client version 1.3.5
  • .NET version: 4.5.2

Trying to create server on 0.0.0.0 throws exception

EDIT: This exception is wrong. See my next comment.
I'm not sure if this is something I'm doing wrong, but I get an exception when trying to:

var server = new MetricServer("0.0.0.0", 10000); server.Start();

The exception is:
System.ObjectDisposedException: Cannot access a disposed object. Object name: 'System.Net.HttpListener'. at System.Net.HttpListener.CheckDisposed() at System.Net.HttpListener.Stop() at Prometheus.MetricServer.StopInner() in C:\MyData\dev\prometheus-net\prometheus-net.shared\MetricServer.cs:line 74 at Prometheus.MetricHandler.Stop() in C:\MyData\dev\prometheus-net\prometheus-net.shared\MetricHandler.cs:line 38

Is this something I'm doing wrong or is it a bug?

Thanks in advance,
Marcelo.

Adding prometheus-net to my project adds over 100 dependent files

I have a .NET 4.6.1 project (class library) that I had previously used with the 1.x version of prometheus-net. When I updated to the 2.x prerelease, over 100 dependent files are added to the project output.

My project doesn't itself target .NET Standard, it targets the .NET Framework.

Thoughts on supporting the ability to turn this on or off?

I'm starting to use this in our WebAPI applications using OWIN, and so far it's working nicely. I'm instrumenting both the OWIN pipeline and our internal API code, but I'm finding myself wanting the ability to toggle this off or on via some configuration. Let's say that for whatever reason I want to suddenly turn off instrumentation all together, have you thought of how you might address that with this library?

Given how pervasive the instrumentation code itself is, one though I had was to support providing a "null" version of the metric related objects that provide the required methods for all the calling code, but just don't do anything. However, this would (at the very least) require interfaces around the objects to support injecting alternate versions (the real one vs the null on based on some condition). Right now I'm using an instance of the MetricFactory in my code to get counters, histograms, etc, so I'd need to provide an alternate version of the factory first that in-turn provided alternate versions of the Counter, Histogram, etc classes.

Have you come across the need for this, or thought about how you might address this scenario?

not thread safe

when the library is used in parallel the /metrics endpoint is iterating over a collection of metrics and other threads might be adding to the collection causing a collection modified while iterating exception.

Push Gateway support

Hi,

We are considering using your library to instrument a B2C .NET Desktop client. I believe we will need to be able to use the Push To Gateway mechanism since our users application sessions are short lived and we cannot expect the prometheus server to scrape hundreds of thousands of clients across our customer base. Do you have any plans to add support for Push To Gateway in this library?

Andrew

Is this in-memory only?

Hello.
I've just found this library yesterday and have a quick question.
I have a console application running and I would like to publish some metrics about it to Prometheus.
That means I should host both the metrics server and my program in the same process?
Also, is this correct that when using this library's api, I only change metrics data in memory, and then Prometheus server asks periodically if I have any updates?
Sorry if it's a dumb question

Can we delete metrics created in the client?

I have created a number of metrics like histograms and counters with different combination of labels. However, it looks that the scrape endpoint after a period of time is shooting up its latency due to the long list of metrics that's being scaped. Can we delete these metrics as a way to reset them at that point in order to manage these metrics better? Or, is there another way to handle this problem from the client?

Summary metrics can return NaN

Needs to be reviewed - does this make sense? If you observe a NaN value by accident, you will contaminate a lot of calculations with the NaN, basically making the summary worthless.

An error has occurred during metrics collection:500 internal server err

prometheus/pushgateway#191 #

When I am trying to use Pushgateway (c#) to publish metrics, I get below err

2 error(s) occurred:

  • gathered metric family process_start_time_seconds has help "Start time of the process since unix epoch in seconds" but should have "Start time of the process since unix epoch in seconds."
  • gathered metric family process_cpu_seconds_total has help "Total user and system CPU time spent in seconds" but should have "Total user and system CPU time spent in seconds."

In my prometheus localhost:9090 dashboard I see the status as 500 intrnl err

http://localhost:9091/metrics | DOWN | instance="localhost:9091" | 581ms ago | server returned HTTP status 500 Internal Server Error

-- I cloned and checked in the code (DotNerStatsCollector.cs) class I saw

        _startTime = metrics.CreateGauge("process_start_time_seconds", "Start time of the process since unix epoch in seconds");
        _cpuTotal = metrics.CreateCounter("process_cpu_seconds_total", "Total user and system CPU time spent in seconds");

here the arguments dont have period in the end !!!!!
So, is period the culprit ??? Am I running uncompatable versions ? is it a bad unit test ???

how to pass through this issue.

Pick a better name for ASP.NET Core middleware activation method

Currently it is called as app.UsePrometheusServer() which is ambiguous - what is a Prometheus server? I would say the database. Perhaps UseMetricServer() might be more straight to the point?

Consider also that there may be at some point ASP.NET Core metrics integration in Prometheus directly, so there should be allowances for a future "use Prometheus to track ASP.NET Core metrics" middleware as well as the current "expose a Prometheus exporter" middleware.

Flush metrics when stopping MetricsPusher

By default metrics are sent to push gateway every second. Therefore, a short job taking less than one second won't have a chance to push anything. I believe Stop() should flush local metrics cache.

Pull request: #34

ASP.NET Core middleware

Hello guys,

I'm not a developer:) I just trying to build POC with prometheus support

I create new project in VS2017 (ASP net core 2.0)

I've installed prometheus-net and prometheus-net.AspNetCore preview from official NuGet server

than i've edited Program.cs

from:

        public static IWebHost BuildWebHost(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
                .UseStartup<Startup>()
                .Build();

to:

using Prometheus;
...
        public static IWebHost BuildWebHost(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
                .Configure(app => app.UseMetricServer())
                .UseStartup<Startup>()
                .Build();

Build is OK. But I couldn't get /metrics (HTTP ERROR 404)

Do I have to add something else?

Add prefix for sample metrics

Hi,

The documentation says that the sample metrics can be removed by calling DefaultCollectorRegistry.Instance.Clear(). Is there way to add a prefix for those metrics?
E.g. my app is called "MyApp", I would like to have one of those metrics called like: myapp_process_windows_processid.

Thanks.

Metric pusher fails with .net standard package

I have used the sample test code given for metric pusher and used the .net standard version of the package and it failed with the following message when i try to access metrics url.

An error has occurred:

expected counter in metric process_cpu_seconds_total label:<name:"instance" value:"App_Name" > label:<name:"job" value:"App_Domain" > gauge:<value:0.296875 > 

it works fine if i change the package to full framework version

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.