wrengr / bytestring-lexing Goto Github PK

View Code? Open in Web Editor NEW

6.0 6.0 4.0 1.71 MB

Efficiently parse and produce common lexemes: namely integral and fractional numbers.

License: Other

Haskell 96.30% CSS 2.24% HTML 1.46%

bytestring-lexing's People

Contributors

Stargazers

Watchers

Forkers

aske juspay forketyfork parsonsmatt

bytestring-lexing's Issues

Improve coverage

HPC currently shows our test suite only has coverage of: 77% (34/44) TLD, 65% (135/205) alternatives, 75% (1222/1616) expressions. Much of the missing coverage is low-hanging fruit (e.g., does Data.ByteString.Lex.Fractional. readDecimal agree with Data.ByteString.Lex.Integral.readDecimal when the latter returns Nothing or when the prefix has no decimal part)

Build failure in v0.3.0 with GHC 7.10.3

dist/build/Data/ByteString/Lex/Lazy/Double.hs:339:17:
    Pattern bindings containing unlifted types should use an outermost bang pattern:
      ((I# (ord_c))) = ord c
    In the expression:
      let
        (base) = alexIndexInt32OffAddr alex_base s
        ((I# (ord_c))) = ord c
        (offset) = (base +# ord_c)
        ....
      in
        case new_s of {
          -1# -> (new_acc, input)
          _ -> alex_scan_tkn
                 user orig_input (len +# 1#) new_input new_s new_acc }
    In a case alternative:
        Just (c, new_input)
          -> let
               (base) = alexIndexInt32OffAddr alex_base s
               ((I# (ord_c))) = ord c
               ....
             in
               case new_s of {
                 -1# -> (new_acc, input)
                 _ -> alex_scan_tkn
                        user orig_input (len +# 1#) new_input new_s new_acc }
    In the second argument of ‘seq’, namely
      ‘case alexGetChar input of {
         Nothing -> (new_acc, input)
         Just (c, new_input)
           -> let
                (base) = ...
                ....
              in
                case new_s of {
                  -1# -> ...
                  _ -> alex_scan_tkn
                         user orig_input (len +# 1#) new_input new_s new_acc } }’

dist/build/Data/ByteString/Lex/Lazy/Double.hs:343:31:
    Couldn't match expected type ‘Bool’ with actual type ‘Int#’
    In the first argument of ‘(&&)’, namely ‘(offset >=# 0#)’
    In the expression: (offset >=# 0#) && (check ==# ord_c)
    In the expression:
      if (offset >=# 0#) && (check ==# ord_c) then
          alexIndexInt16OffAddr alex_table offset
      else
          alexIndexInt16OffAddr alex_deflt s

dist/build/Data/ByteString/Lex/Lazy/Double.hs:343:50:
    Couldn't match expected type ‘Bool’ with actual type ‘Int#’
    In the second argument of ‘(&&)’, namely ‘(check ==# ord_c)’
    In the expression: (offset >=# 0#) && (check ==# ord_c)
    In the expression:
      if (offset >=# 0#) && (check ==# ord_c) then
          alexIndexInt16OffAddr alex_table offset
      else
          alexIndexInt16OffAddr alex_deflt s
cabal: Failed to build bytestring-lexing-0.3.0

I encountered this error while building hedis-0.3.2, with the following build plan:

$ cabal install hedis-0.3.2 -w ghc-7.10.3
Resolving dependencies...
Build profile: -w ghc-7.10.3 -O1
In order, the following will be built (use -v for more details):
 - alex-3.1.4 (exe:alex) (requires download & build)
 - bytestring-0.9.2.1 (lib) (requires download & build)
 - stm-2.2.0.1 (lib:stm) (requires download & build)
 - transformers-base-0.4.2 (lib:transformers-base) (requires download & build)
 - unix-2.7.2.2 (lib:unix) (requires build)
 - bytestring-lexing-0.3.0 (lib:bytestring-lexing) (requires download & build)
 ...

For some reason this failure does not show up in the Hackage Matrix builder: https://matrix.hackage.haskell.org/#/package/bytestring-lexing

I assume the dependency configuration is somehow different there.

Update instructions for running tests

Add support for partial input

This issue tracks recently discussion on haskell cafe. maybe we can add something like:

data Result a = Partail (ByteString -> Result a) | Done a ByteString | Failure

readDecimalPartial :: Integral a => ByteString -> Result a

But this seems too complex for a lexer, i will try to improve binary-parsers to see if i can manage a lexer loop there first.

CI failing for GHC 9.0.1

The GithubActions CI is failing for GHC 9.0.1 because cabal sdist isn't creating the expected tarball for cabal test to use. These commands work just fine for me on my local machine with that version, and for GithubActions for older GHC/Cabal, so I've no idea what's going wrong...

Move benchmarks to the main cabal file

Add fast lexers for Integer and Natural

Bertram Felgenhauer has written one for strings that should translate well for decimal conversion.

byteArrayToBigNat# should let you write something absurdly fast for hexadecimal and (if you think it's worth the trouble) octal. Count the digits to determine how many bytes you need, round up to the nearest multiple of the word size, allocate a MutableByteArray#, fill it up with words (I think) smallest first, unsafe freeze it, and then convert for free. Special case numbers that fit in one word.

Performance regression in packDecimal/numDecimalDigits

The current version of packDecimal shows a 3x slowdown compared to the version in the benchmark. This is due to the n0 > fromIntegral (maxBound :: Word64) guard in numDecimalDigits. This must be corrected. a 1.2x slowdown may be acceptable, but 3x is not.

Native support for bytestring >=0.11

bytestring-0.11.0.0 (September 2020) removed the offset parameter to the ByteString data constructor: haskell/bytestring#175 . They provide a PS pattern-synonym for backward compatibility (https://github.com/haskell/bytestring/blob/master/Changelog.md) and while CI indicates that's sufficient for now, we should surely update things to use the new internal representation directly.

$ cabal build
Error: cabal: No targets given and there is no package in the current
directory. Use the target 'all' for all packages in the project or specify
packages or components by name or location. See 'cabal build --help' for more
details on target options.

When using the v1 command

$ cabal v1-configure
Resolving dependencies...
Warning: solver failed to find a solution:
Could not resolve dependencies:
[__0] trying: bytestring-lexing-bench-0.11.0.1 (user goal)
[__1] unknown package: scientific (dependency of bytestring-lexing-bench)
[__1] fail (backjumping, conflict set: bytestring-lexing-bench, scientific)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: bytestring-lexing-bench, scientific
Trying configure anyway.
Configuring bytestring-lexing-bench-0.11.0.1...
Error: cabal: Encountered missing or private dependencies:
QuickCheck >=2.4.1,
attoparsec >=0.13.0.0,
bytestring-lexing >=0.5.0,
bytestring-read >=0.3.0,
criterion >=0.5.0,
scientific >=0.3.3.8

Support GHC-9.6

Seems like bytestring-lexing rejects base-4.18:

[__4] next goal: bytestring-lexing (dependency of hedis)
[__4] rejecting: bytestring-lexing-0.5.0.9 (conflict: bytestring =>
base==4.18.0.0/installed-4.18.0.0, bytestring-lexing => base>=4.5 && <4.18)

Allow-newer semes to work, so maybe a revision is sufficient to fix this.