mdom / dategrep Goto Github PK

View Code? Open in Web Editor NEW

174.0 6.0 12.0 1.75 MB

print lines matching a time range

License: Other

Perl 96.23% Shell 3.40% Raku 0.37%

command-line-tool unix grep perl timestamp daterange logging

dategrep's People

Contributors

Stargazers

Watchers

Forkers

gagneet szepeviktor sband javajavelin rameezts shekkbuilder fengsmith qu884j ktosiu yamlcase carlvine500

dategrep's Issues

standalone-big does not contain parent.pm

When trying to run dategrep-standalone-big I get this:

./dategrep
Can't locate parent.pm in @INC (@INC contains: FatPacked::8076224=HASH(0x7b3bc0) /usr/lib/perl5/5.10.0/x86_64-linux-thread-multi /usr/lib/perl5/5.10.0 /usr/lib/perl5/site_perl/5.10.0/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.10.0 /usr/lib/perl5/vendor_perl/5.10.0/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.10.0 /usr/lib/perl5/vendor_perl .) at ./dategrep line 13.
BEGIN failed--compilation aborted at ./dategrep line 13.
Compilation failed in require at ./dategrep line 9.
BEGIN failed--compilation aborted at ./dategrep line 9.
Compilation failed in require at ./dategrep line 4016.
BEGIN failed--compilation aborted at ./dategrep line 4016.

My understanding is that the standalone-big packaging should contain the dependencies?

This was observed on a SUSE Linux Enterprise Server 11 SP3 machine. The workaround is to install perl-parent from "devel languages perl" on download.opensuse.org. Do so with zypper ar http://download.opensuse.org/repositories/devel:/languages:/perl/SLE_11_SP3/devel:languages:perl.repo and then zypper in perl-parent.

Fix pod errors on cpan

Either remove the pod sections or move the functions to the Iterator Base class

Unparsable line

Hi,

i have lines like this in my syslog:

Feb 07 20:24:33 [vdr] [4148] [plex]: 'ChannelSwitch'
- Last output repeated 3 times -
Feb 07 20:24:35 [vdr] [5014] [plex]: Detected client discovery request from 192.168.177.10
- Last output repeated twice -
Feb 07 20:24:43 [vdr] [4168] frontend 1/0 timed out while tuning to channel 0 (), tp 112539

But it seems, datagrep dont like this?

vdr01_64 ~ # ./dategrep-standalone-big.pl --start "12:00" --end "12:15" --format "%b %d %H:%M:%S" /log/messages
dategrep-standalone-big.pl: Unparsable line: - Last output repeated twice -
vdr01_64 ~ #

0.55

Hello!

Do you plan to solve some issues and release the next version?

Add information to debian/changelog

Originally posted by @szepeviktor in #32 (comment)

Infinite loop

@mdom Good morning!

I have an infinite loop on this request. (one line)
Please help me.

217.13.101.170 - - [24/Nov/2015:18:41:13 +0100] "GET /https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunhttps://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunhttps://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunkaido.hu/https://olmunk" 414 0 "-" "-"

Installation

Please tell rookies in the readme:

cpan App:datagrep

dategrep-standalone-big reports wrong version

I downloaded version 0.57 from https://github.com/mdom/dategrep/releases and ran ./dategrep-standalone-big --version. It reports 0.56.

Move iteratator functions from dategrep.pm to Iterator.pm

Package as a cpan module App::dategrep

would make it much easier to install.

cache next date on file for interleave

and not just reparse the date on every sort

Guess format for every filehandle

Drop Milla and use plain DZIL

I changed so much in the milla distribution, i could probably just drop it and use plain dzil.

Evaluate Gzip::RandomAccess

Debian rsyslog timestamp format

Good morning!

How could it be that Debian's default $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat which looks like this Jun__8_15:25:22 can be parsed with your "%b %d %H:%M:%S"?
I've replaces spaces with underscores.

Fixed download URL gone

Please help users identifying the latest version.

release="$(wget -qO- https://api.github.com/repos/mdom/dategrep/releases|sed -n '0,/^.*"tag_name": "\([0-9.]\+\)".*$/s//\1/p')"

https://github.com/mdom/dategrep/blob/master/README.pod#installation

Fails with German locale

As per subject. Sample fail report:

http://www.cpantesters.org/cpan/report/97271888

Statistics: http://analysis.cpantesters.org/reports_by_field?distv=dategrep-2.00;field=env%3ALC_ALL

add --ignore-unparsable

deactivates --multiline and just skips unparsable lines

Gzip support with IO::Uncompress::Gunzip

don't rely on a existing gzip

dockerize dategrep and potentially distribute to hub.docker.com

Would you consider providing a dockerfile for building a dategrep on perl docker image? We can do that for ourselves of course, but I really like getting the author's idea of what source image to use and what should go in it.

Extra Credit: Build a pipeline and have the image pushed up to docker hub with each passing commit.

--format should be a required parameter

peek_line vs peek_entry

I probable need both functions

build debian package for release

Cleanly seperate readme files

README.md should just introduce the project, lists more examples, handle installation from github and link to the man page. The manpage shouldn't list installation instruction and INSTALLATION should just be the perl boilerplate

Please consider tagged releases

https://help.github.com/articles/creating-releases/

./Build standalone problem

perl Build.PL
WARNING: the following files are missing in your kit:
        Makefile.PL
        META.json
        META.yml
Please inform the author.

Created MYMETA.yml and MYMETA.json
Creating new 'Build' script for 'App-dategrep' version '0.14'

./Build standalone

./dategrep-standalone --format rsyslog --multiline --from "1 hour ago from -17:00" --to "-17:00" /var/log/syslog
Undefined subroutine &App::dategrep::loadconfig called at ./dategrep-standalone line 77.

Invalid lines

Could there be an option to skip invalid lines, lines without date?
Now I use sed -e '1!b;/^(Nothing .*$/d'

My boot.log

(Nothing has been logged yet.)
Thu Dec 17 00:04:36 2015: [....] Checking root file system...fsck from util-linux 2.25.2

Implementing intervals

Thank you!

Microseconds in Apache error log

Cloud you help me to match [Mon Jul 06 06:45:16.915695 2015]?

%a %b %d %T.?????? %Y

New release

@mdom Please consider releasing those 35 commits.
https://github.com/mdom/dategrep/releases

Accurate to seconds

With the latest version, filtering is limited to the minute level, not the exact second level

And it only filters up to the last minute. Down

~]$ echo $pre $cur
11:58:12 12:02:30

~]$ ./dategrep/dategrep --format "%H:%M:%S" --start $pre --end $cur ./access.log.1 > grep.log

~]$ head -1 grep.log
[2020-02-24 11:58:12,007] ..........

~]$ ~]$ tail -1 grep.log #12:01 and 12:02 and a lot more data
[2020-02-24 12:00:01,494] ..........

Test suite started to fail in 2019

Test suite started to fail today (2019-01-01):

#   Failed test 'Mon Jul 02 -> %a %b %d'
#   at t/11strptime.t line 21.
#          got: '1562025600'
#     expected: '1530489600'

#   Failed test 'Mon Jul  2 -> %a %b %d'
#   at t/11strptime.t line 21.
#          got: '1562025600'
#     expected: '1530489600'
# Looks like you failed 2 tests of 14.
t/11strptime.t ......... 
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/14 subtests

Smaller date library

use Time::Piece and write a small wrapper for the most important date formats and expressing deltas. Provide Date::Manip as an alternative backend

Express "1 day ago" after replacing Date::Manip

Could it be that --start "now truncate 48h add 6h25m" --end "06:25:00" means from yesterday's 6:25 to today's 6:25?

Thank you.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.