zezax / one Goto Github PK

2.0 7.0 0.0 719 KB

This is the "one": Linuxy infrastructureish mostly C++ code.

License: MIT License

Python 13.55% Shell 3.21% Makefile 0.94% C++ 82.30%

automation deterministic-finite-automata dfa dfa-construction dfa-minimization linux log-monitor regex-engine regular-expression rrd

one's People

Contributors

Stargazers

Watchers

one's Issues

Match API That Saves State Across Calls

This would be useful when the text arrives in installments and/or is too large to fit in memory all at once.

ASAN error in DefaultMap (Spurious?)

Seen on Debian with gcc 10.2.1-6...

$ make
$ make -C test test-defaultmap
...

==46129==ERROR: AddressSanitizer: heap-use-after-free on address 0x6020000001f8 at pc 0x555f2a6d9a25 bp 0x7ffe8b65dcf0 sp 0x7ffe8b65dce8
READ of size 4 at 0x6020000001f8 thread T0
    #0 0x555f2a6d9a24 in std::__detail::_Hash_code_base<int, std::pair<int const, int>, std::__detail::_Select1st, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, false>::_M_bucket_index(std::__detail::_Hash_node<std::pair<int const, int>, false> const*, unsigned long) const /usr/include/c++/10/bits/hashtable_policy.h:1307
    #1 0x555f2a6d8087 in std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_bucket_index(std::__detail::_Hash_node<std::pair<int const, int>, false>*) const /usr/include/c++/10/bits/hashtable.h:678
    #2 0x555f2a6d6da2 in std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_move_assign(std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >&&, std::integral_constant<bool, true>) /usr/include/c++/10/bits/hashtable.h:1238
    #3 0x555f2a6d5167 in std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::operator=(std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >&&) /usr/include/c++/10/bits/hashtable.h:519
    #4 0x555f2a6d3b7c in std::unordered_map<int, int, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, int> > >::operator=(std::unordered_map<int, int, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, int> > >&&) /usr/include/c++/10/bits/unordered_map.h:275
    #5 0x555f2a6d3ba6 in zezax::red::DefaultMap<int, int>::operator=(zezax::red::DefaultMap<int, int>&&) ../include/DefaultMap.h:10
    #6 0x555f2a6d26f5 in DefaultMap_smoke_Test::TestBody() /home/ameyer/github/one/quol/red/test/defaultmap.cpp:29
    #7 0x555f2a7088b6 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/home/ameyer/github/one/quol/red/_build/test/defaultmap+0x448b6)
    #8 0x555f2a6fd42d in testing::Test::Run() (/home/ameyer/github/one/quol/red/_build/test/defaultmap+0x3942d)
    #9 0x555f2a6fd584 in testing::TestInfo::Run() (/home/ameyer/github/one/quol/red/_build/test/defaultmap+0x39584)
    #10 0x555f2a6fda18 in testing::TestSuite::Run() (/home/ameyer/github/one/quol/red/_build/test/defaultmap+0x39a18)
    #11 0x555f2a6fe061 in testing::internal::UnitTestImpl::RunAllTests() (/home/ameyer/github/one/quol/red/_build/test/defaultmap+0x3a061)
    #12 0x555f2a708e26 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (/home/ameyer/github/one/quol/red/_build/test/defaultmap+0x44e26)
    #13 0x555f2a6fd647 in testing::UnitTest::Run() (/home/ameyer/github/one/quol/red/_build/test/defaultmap+0x39647)
    #14 0x555f2a6d0f0f in main (/home/ameyer/github/one/quol/red/_build/test/defaultmap+0xcf0f)
    #15 0x7fbea63abd09 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x23d09)
    #16 0x555f2a6d1379 in _start (/home/ameyer/github/one/quol/red/_build/test/defaultmap+0xd379)

0x6020000001f8 is located 8 bytes inside of 16-byte region [0x6020000001f0,0x602000000200)
freed by thread T0 here:
    #0 0x7fbea6955467 in operator delete(void*, unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:172
    #1 0x555f2a6dbb1f in __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> >::deallocate(std::__detail::_Hash_node<std::pair<int const, int>, false>*, unsigned long) /usr/include/c++/10/ext/new_allocator.h:133
    #2 0x555f2a6daa38 in std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> >::deallocate(std::__detail::_Hash_node<std::pair<int const, int>, false>*, unsigned long) /usr/include/c++/10/bits/allocator.h:187
    #3 0x555f2a6daa38 in std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> > >::deallocate(std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> >&, std::__detail::_Hash_node<std::pair<int const, int>, false>*, unsigned long) /usr/include/c++/10/bits/alloc_traits.h:492
    #4 0x555f2a6d9acc in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> > >::_M_deallocate_node_ptr(std::__detail::_Hash_node<std::pair<int const, int>, false>*) /usr/include/c++/10/bits/hashtable_policy.h:2064
    #5 0x555f2a6d8117 in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> > >::_M_deallocate_node(std::__detail::_Hash_node<std::pair<int const, int>, false>*) /usr/include/c++/10/bits/hashtable_policy.h:2054
    #6 0x555f2a6d62b3 in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> > >::_M_deallocate_nodes(std::__detail::_Hash_node<std::pair<int const, int>, false>*) /usr/include/c++/10/bits/hashtable_policy.h:2075
    #7 0x555f2a6d6a9c in std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_move_assign(std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >&&, std::integral_constant<bool, true>) /usr/include/c++/10/bits/hashtable.h:1219
    #8 0x555f2a6d5167 in std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::operator=(std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >&&) /usr/include/c++/10/bits/hashtable.h:519
    #9 0x555f2a6d3b7c in std::unordered_map<int, int, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, int> > >::operator=(std::unordered_map<int, int, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, int> > >&&) /usr/include/c++/10/bits/unordered_map.h:275
    #10 0x555f2a6d3ba6 in zezax::red::DefaultMap<int, int>::operator=(zezax::red::DefaultMap<int, int>&&) ../include/DefaultMap.h:10
    #11 0x555f2a6d26f5 in DefaultMap_smoke_Test::TestBody() /home/ameyer/github/one/quol/red/test/defaultmap.cpp:29
    #12 0x555f2a7088b6 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/home/ameyer/github/one/quol/red/_build/test/defaultmap+0x448b6)

previously allocated by thread T0 here:
    #0 0x7fbea6954647 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
    #1 0x555f2a6dbaee in __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> >::allocate(unsigned long, void const*) /usr/include/c++/10/ext/new_allocator.h:115
    #2 0x555f2a6da881 in std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> >::allocate(unsigned long) /usr/include/c++/10/bits/allocator.h:173
    #3 0x555f2a6da881 in std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> > >::allocate(std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> >&, unsigned long) /usr/include/c++/10/bits/alloc_traits.h:460
    #4 0x555f2a6d991b in std::__detail::_Hash_node<std::pair<int const, int>, false>* std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> > >::_M_allocate_node<std::pair<int const, int> const&>(std::pair<int const, int> const&) /usr/include/c++/10/bits/hashtable_policy.h:2032
    #5 0x555f2a6d801d in std::__detail::_Hash_node<std::pair<int const, int>, false>* std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> > >::operator()<std::pair<int const, int> const&>(std::pair<int const, int> const&) const /usr/include/c++/10/bits/hashtable_policy.h:167
    #6 0x555f2a6d5fb7 in void std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_assign<std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> > const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> > > >(std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> > const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::pair<int const, int>, false> > > const&) /usr/include/c++/10/bits/hashtable.h:1166
    #7 0x555f2a6d4ce6 in std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_Hashtable(std::_Hashtable<int, std::pair<int const, int>, std::allocator<std::pair<int const, int> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> > const&) /usr/include/c++/10/bits/hashtable.h:1279
    #8 0x555f2a6d3ad2 in std::unordered_map<int, int, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, int> > >::unordered_map(std::unordered_map<int, int, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, int> > > const&) /usr/include/c++/10/bits/unordered_map.h:181
    #9 0x555f2a6d3af8 in zezax::red::DefaultMap<int, int>::DefaultMap(zezax::red::DefaultMap<int, int> const&) ../include/DefaultMap.h:10
    #10 0x555f2a6d2258 in DefaultMap_smoke_Test::TestBody() /home/ameyer/github/one/quol/red/test/defaultmap.cpp:25
    #11 0x555f2a7088b6 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/home/ameyer/github/one/quol/red/_build/test/defaultmap+0x448b6)

SUMMARY: AddressSanitizer: heap-use-after-free /usr/include/c++/10/bits/hashtable_policy.h:1307 in std::__detail::_Hash_code_base<int, std::pair<int const, int>, std::__detail::_Select1st, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, false>::_M_bucket_index(std::__detail::_Hash_node<std::pair<int const, int>, false> const*, unsigned long) const
Shadow bytes around the buggy address:
  0x0c047fff7fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c047fff7ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c047fff8000: fa fa 00 fa fa fa 00 00 fa fa 00 fa fa fa 00 fa
  0x0c047fff8010: fa fa 00 fa fa fa 00 fa fa fa 00 fa fa fa 04 fa
  0x0c047fff8020: fa fa 00 fa fa fa 04 fa fa fa 00 fa fa fa 00 00
=>0x0c047fff8030: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa fd[fd]
  0x0c047fff8040: fa fa fd fd fa fa fd fd fa fa fa fa fa fa fa fa
  0x0c047fff8050: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8060: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8070: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff8080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==46129==ABORTING
make: *** [Makefile:24: test-defaultmap] Error 1
make: Leaving directory '/home/ameyer/github/one/quol/red/test'

Red: Estimate Size of Classes

Here's a breakdown of significant lines of code for each class concept. It combines .h, .cpp, and test code from the include, lib and test directories respectively. The Total column should give a rough idea of how much work it is to review.

Class	Header	CPP	Test	Total	Type	Notes
BitSet	379	0	299	678	template, bit twiddling
Budget	24	0	0	24
Compile	10	33	0	43	wrapper
Consts	16	0	0	16
Debug	74	495	156	725		skip review
DefaultMap	29	0	25	54	template
Dfa	134	220	348	702	iterators, algorithm
Except	55	0	0	55
Executable	45	142	77	264	container
Fnv	31	0	15	46	template
Matcher	583	62	675	1320	templates, optimized
Minimizer	118	256	156	530	algorithm
Nfa	148	269	83	500	iterators, operations
Outcome	21	0	0	21
omnibus	0	0	351	351		just tests
Parser	34	185	75	294	tricky
Powerset	42	171	117	330	algorithm
Proxy	117	0	0	117	iterators, template
Red	169	482	449	1100	API	much generated
Scanner	48	262	193	503	tricky
Serializer	67	233	73	373	file format
SparseVec	58	0	110	168	template, algorithm
Types	77	0	0	77
Util	45	129	74	248

The Debug stuff isn't used by the library; so, bugs there wouldn't be very concerning. It's used in some unit tests and is available for ad hoc debugging.

Serializer.cpp compilation failure

Ubuntu 20.04.2, gcc version 10.3.0

g++ -c -MMD -MT ../_build/lib/Serializer.o -MT ../_build/lib/Serializer.d -std=c++20 -g -fsanitize=address -Wall -Wextra -Wshadow -Wpointer-arith -Wwrite-strings -Wsign-promo -Wuninitialized -Wconversion -Wfloat-equal -Wrestrict -Wdisabled-optimization -Wduplicated-branches -Wduplicated-cond -Wformat=2 -Wlogical-op -Wnull-dereference -Woverloaded-virtual -Wuseless-cast -Wimplicit-fallthrough=2 -Wmissing-include-dirs -Werror -iquote ../include Serializer.cpp -o ../_build/lib/Serializer.o
In file included from Serializer.cpp:10:
../include/Proxy.h: In instantiation of ‘static zezax::red::DfaProxy::Value zezax::red::DfaProxy::resultAndDeadEnd(zezax::red::Result, bool) [with zezax::red::Format fmt = zezax::red::fmtDirect1; zezax::red::DfaProxy::Value = unsigned char; zezax::red::Result = int]’:
Serializer.cpp:40:49: required from ‘void zezax::red::{anonymous}::appendStateImpl(std::string&, const zezax::red::DfaState&, zezax::red::CharIdx, const std::vector<long unsigned int, std::allocator >&) [with zezax::red::Format fmt = zezax::red::fmtDirect1; std::string = std::cxx11::basic_string; zezax::red::CharIdx = unsigned int]’
Serializer.cpp:201:60: required from here
../include/Proxy.h:166:32: error: conversion from ‘zezax::red::Result’ {aka ‘int’} to ‘zezax::red::DfaProxyzezax::red::fmtDirect1::Value’ {aka ‘unsigned char’} may change value [-Werror=conversion]
166 | return (res & resultMask) | (de << deadEndShift);
| ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
../include/Proxy.h: In instantiation of ‘static zezax::red::DfaProxy::Value zezax::red::DfaProxy::resultAndDeadEnd(zezax::red::Result, bool) [with zezax::red::Format fmt = zezax::red::fmtDirect2; zezax::red::DfaProxy::Value = short unsigned int; zezax::red::Result = int]’:
Serializer.cpp:40:49: required from ‘void zezax::red::{anonymous}::appendStateImpl(std::string&, const zezax::red::DfaState&, zezax::red::CharIdx, const std::vector<long unsigned int, std::allocator >&) [with zezax::red::Format fmt = zezax::red::fmtDirect2; std::string = std::__cxx11::basic_string; zezax::red::CharIdx = unsigned int]’
Serializer.cpp:204:60: required from here
../include/Proxy.h:166:32: error: conversion from ‘zezax::red::Result’ {aka ‘int’} to ‘zezax::red::DfaProxyzezax::red::fmtDirect2::Value’ {aka ‘short unsigned int’} may change value [-Werror=conversion]
cc1plus: all warnings being treated as errors
make[1]: *** [../red.gmake:70: ../_build/lib/Serializer.o] Error 1
make[1]: Leaving directory '/home/john/web/zezax/quol/red/lib'
make: *** [Makefile:26: all-lib] Error 2

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.