Coder Social home page Coder Social logo

Improve report query time about anemometer HOT 2 CLOSED

box avatar box commented on August 16, 2024
Improve report query time

from anemometer.

Comments (2)

gtowey avatar gtowey commented on August 16, 2024

Note that this will require instructions on how to upgrade for existing users, because it requires a change in table schema, and a change in arguments to pt-query-digest. Roughly those steps are:

  1. change pt-query-digest --filter argument
pt-query-digest  \
--filter=' $event->{Bytes} = length($event->{arg}) and $event->{hostname}="app-db2003.sv2.box.net:3306" and $event->{Ts_date}=parse_timestamp(substr($event->{ts},0,6)." 00:00:00") and $event->{Ts_hour}=parse_timestamp(substr($event->{ts},0,9).":00:00") and $event->{Ts_minute}=parse_timestamp(substr($event->{ts},0,12).":00")'
  1. change schema
ALTER TABLE global_query_review_history 
 ADD ts_date_min date, add ts_date_max date, 
 ADD ts_hour_min datetime,
 ADD ts_hour_max datetime,
 ADD ts_minute_min datetime,
 ADD ts_minute_max datetime,
 DROP key checksum,
 ADD key `history` (checksum, ts_date_min),
 ADD key `graph_query` (checksum,ts_minute_min),
 ADD key (ts_minute_min);
  1. backfill data
UPDATE global_query_review_history_index SET
 ts_date_min=DATE(ts_min), ts_date_max=DATE(ts_max), 
 ts_hour_min=FROM_UNIXTIME(TRUNCATE(UNIX_TIMESTAMP(ts_min)/3600,0)*3600),
 ts_hour_max = FROM_UNIXTIME( TRUNCATE(UNIX_TIMESTAMP(ts_max) / 3600,0)*3600),
 ts_minute_min=FROM_UNIXTIME( TRUNCATE(UNIX_TIMESTAMP(ts_min) / 60,0)*60),
 ts_minute_max=FROM_UNIXTIME( TRUNCATE(UNIX_TIMESTAMP(ts_max) /60,0)*60);

from anemometer.

ankitthakwani avatar ankitthakwani commented on August 16, 2024

The bacfill data UPDATE SQL refers to table global_query_review_history_index, but only 2 tables exist in the DB

global_query_review
global_query_review_history

Please let me know if I am interpreting something incorrectly.

from anemometer.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.