badoo / liveprof-ui

An aggregator and web interface for Live Profiler

Live Profiler UI

Live profiler is a system-wide performance monitoring system in use at Badoo that is built on top of XHProf or its forks (Uprofiler or Tideways). Live Profiler continually gathers function-level profiler data from production tier by running a sample of page requests under XHProf.

Live profiler UI aggregates the profile data corresponding to individual requests by various dimensions such a time, memory usage, and can help answer a variety of questions such as: What is the function-level profile for a specific page? How expensive is function "foo" across all pages, or on a specific page? What functions regressed most in the last day/week/month? What is the historical trend for execution time of a page/function? and so on.

You can find the full documentation in Live Profiler UI wiki

Here is a plugin for PhpStorm to see the method performance directly in IDE.

System Requirements

• PHP version 7.3 or later to use web interface and run aggregation scripts.
• PHP version 5.4 or later / hhvm version 3.25.0 or later to collect profiles using Live Profiler
• Connection to database with profiling result. You can collect profiles using Live Profiler tool
• Database extension (mysqli, pgsql, sqlite support included)
• Perl for flame graph functionality

Key features

• Get stats of average value, minimum, maximum, 95 percentile of execution time, cpu time, memory usage and calls count. Parameter list and statistics functions are configurable.
• Graphs for every collected parameter and every method up to 6 months. Each graph also includes children stats. It helps to see the history of changes.
• Differences interface to compare a particular request for two dates and see what became worse.
• See flame graph of the aggregated request.
• Get list of requests where a method was called last time. It may be helpful for refactoring purposes and find unused methods.
• Get the most changed methods in any requests for two dates, for example, today and a week ago. It can help to find a place of a potential performance problem.

Installation guide

• Run in the Docker container
• Clone git repository

Work flow

Live Profiler has 3 main parts:

1. Profiler
2. Aggregator
3. Web interface
   • Profile list
   • Methods tree page
   • Method list
   • Snapshots comparison interface
   • Flame graph
   • Find method usage
   • Most changed snapshots

Performance investigation guide

Customisation

Troubleshooting

License

This project is licensed under the MIT open source license.

liveprof-ui's Issues

Typo in aggregator command

/usr/local/bin/php /app/bin/cli.php aggregator:aggregate-manual app label YYYY-MM-DD
should actually be:
/usr/local/bin/php /app/bin/cli.php aggregator:aggregate-manual label app YYYY-MM-DD
label goes first as it is a required argument

Aggregator halt

Moved from: badoo/liveprof#4

--

Describe the bug
This command runned from cron script:

AGGREGATOR_CONFIG_PATH=/app/docker/php/liveprof.generated.yaml php /app/tools/liveprof-ui/bin/cli.php cron:process-aggregating-jobs

and script fail with error

Fatal error: Uncaught TypeError: rtrim() expects parameter 1 to be string, null given in /app/tools/liveprof-ui/src/Badoo/LiveProfilerUI/Aggregator.php:268
Stack trace:
#0 /app/tools/liveprof-ui/src/Badoo/LiveProfilerUI/Aggregator.php(268): rtrim(NULL, ',')
#1 /app/tools/liveprof-ui/src/Badoo/LiveProfilerUI/Aggregator.php(250): Badoo\LiveProfilerUI\Aggregator->aggregateRow(Array)
#2 /app/tools/liveprof-ui/src/Badoo/LiveProfilerUI/Aggregator.php(189): Badoo\LiveProfilerUI\Aggregator->aggregate()
#3 /app/tools/liveprof-ui/src/Badoo/LiveProfilerUI/ConsoleCommands/ProcessAggregatingJobsCommand.php(60): Badoo\LiveProfilerUI\Aggregator->process()
#4 /app/tools/liveprof-ui/vendor/symfony/console/Command/Command.php(255): Badoo\LiveProfilerUI\ConsoleCommands\ProcessAggregatingJobsCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /app/tools/liveprof-ui/vendor/symfony/console/Application.php(934): Symfony\Component\Console\Command\Command->run(Object(Symfo in /app/tools/liveprof-ui/src/Badoo/LiveProfilerUI/Aggregator.php on line 268

Expected behavior
Script is working

Additional context
Current state of aggregator_jobs table

select status, count(*)
from aggregator_jobs
group by status

This numbers not changing:

[
  {
    "status": "finished",
    "count(*)": 57
  },
  {
    "status": "new",
    "count(*)": 318
  },
  {
    "status": "processing",
    "count(*)": 19
  }
]

How to aggregate via xhprof_samping_enable?

Dear authors LiveProf, thanks you for your efforts
I think use xhprof_enable() as default callstack tracing method have not enough accurancy and require much CPU resources

i realized some custom Liveprof usage via xhprof_sample_enable + xhprof_sample_disable

https://github.com/Slach/howto-sample-profiling-and-flamegraph/blob/master/php/fpm/conf.d/liveprof_sampling.php

it's work with much lower CPU usage and can used for every php request

but when Liveprof work fine, Liveprof-UI aggregate result which not macth what i see in database
please see sql dump with details
https://mega.nz/#!OIpCTCSa!xGfjWqBM55yDoA0yO5iSvpoKoISICC5TbYL4AAzD4g0

Could you explain me why some additional info from details (like SQL queries) haven't parsed?

Place aggregator_processor.lock file in custom folder

Hello! In file src/Badoo/LiveProfilerUI/ConsoleCommands/ProcessAggregatingJobsCommand.php at line 31:
$lock_filename = '/tmp/aggregator_processor.lock';
Would be great to change hardcoded/tmp destination folder by setting it in src/config/aggregator_services.yaml config file, or using system/php variables (like TMP, TMPDIR or TEMP). That's important when i need 2 instances of liveprof/liveprof-ui on same host without docker.

Tnx for this usefull tool!

UPD: same in tests/unit/Badoo/LiveProfilerUI/LoggerTest.php, but i'm not using it =)