vincenthz / hs-asn1 Goto Github PK
View Code? Open in Web Editor NEWASN1 Raw/BER/DER/CER reader/writer in haskell
License: Other
ASN1 Raw/BER/DER/CER reader/writer in haskell
License: Other
Greetings,
Building library for asn1-parse-0.9.4..
[1 of 1] Compiling Data.ASN1.Parse ( Data/ASN1/Parse.hs, dist/dist-sandbox-9df84abc/build/Data/ASN1/Parse.o )
Data/ASN1/Parse.hs:28:1: error:
Could not find module ‘Data.ASN1.Types’
There are files missing in the ‘asn1-types-0.3.2’ package,
try running 'ghc-pkg check'.
Use -v to see a list of the files searched for.
|
28 | import Data.ASN1.Types
| ^^^^^^^^^^^^^^^^^^^^^^
Data/ASN1/Parse.hs:29:1: error:
Could not find module ‘Data.ASN1.Stream’
There are files missing in the ‘asn1-encoding-0.9.5’ package,
try running 'ghc-pkg check'.
Use -v to see a list of the files searched for.
|
29 | import Data.ASN1.Stream
| ^^^^^^^^^^^^^^^^^^^^^^^
cabal: Leaving directory '/tmp/cabal-tmp-1246/asn1-parse-0.9.4'
Thanks for your time.
The x.696 Octet Encoding Rules are a relatively recent (2014) standard for space- and computing-efficient encoding.
In comparison to PER, they trade ease of implementation and en-/decoding complexity for space, so they are a little less space efficient but are supposed to be easier to implement.
As this module doesn't yet have support for any space-efficient packed encodings, this would be a good point to start with.
Is there any interest in supporting this?
While fuzz-testing my code I noticed that although decodeASN1 DER
returns an Either ASN1Error …
, it can throw such errors.
The problem seems to be this code:
instance ASN1Decoding DER where
decodeASN1 _ lbs = (map fst . decodeEventASN1Repr checkDER) `fmap` parseLBS lbs
…
decodeEventASN1Repr :: (ASN1Header -> Maybe ASN1Error) -> [ASN1Event] -> [ASN1Repr]
decodeEventASN1Repr checkHeader l = loop [] l
where loop _ [] = []
loop acc (h@(Header hdr@(ASN1Header _ _ True _)):ConstructionBegin:xs) =
let ctype = decodeConstruction hdr in
case checkHeader hdr of
Nothing -> (Start ctype,[h,ConstructionBegin]) : loop (ctype:acc) xs
Just err -> E.throw err
loop acc (h@(Header hdr@(ASN1Header _ _ False _)):p@(Primitive prim):xs) =
case checkHeader hdr of
Nothing -> case decodePrimitive hdr prim of
Left err -> E.throw err
Right obj -> (obj, [h,p]) : loop acc xs
Just err -> E.throw err
loop (ctype:acc) (ConstructionEnd:xs) = (End ctype, [ConstructionEnd]) : loop acc xs
loop _ (x:_) = E.throw $ StreamUnexpectedSituation (show x)
and the use of E.throw
here.
Shouldn’t decodeASN1
use >>=
instead of fmap
and decodeEventASN1Repr
return an Either ASN1Error …
?
I am observing this:
> decodeASN1 DER (encodeASN1 DER [OID [2,999,2]])
Right [OID [1,15,2]]
Given that
> Text.Hex.encodeHex (Text.Hex.strictByteString (encodeASN1 DER [OID [2,999,2]]))
"06023702"
it seems that the encoder is broken, and the decoder is right…
Data/ASN1/Prim.hs:208:34:
Couldn't match expected type Int' with actual type
Integer'
Expected type: Either ASN1Error Int
Actual type: Either ASN1Error Integer
In the return type of a call of getIntegerRaw' In the second argument of
(<$>)', namely
`getIntegerRaw "enumerated" s'
On Debian Weezy amd64, ghc 7.4.1 .
Noticed via Stackage's Haskell Platform build, which constrains QuickCheck.
$ cabal install asn1-encoding --constraint 'QuickCheck < 2.7' --run-tests --reinstall --force-reinstalls
...
tests/Tests.hs:27:38: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:37:28: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:40:17: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:46:20: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:65:17: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:67:32: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:68:33: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:69:33: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:73:28: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:75:50: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:80:43: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:86:39: Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:130:18:
Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:133:43:
Not in scope: type constructor or class ‘Gen’
tests/Tests.hs:172:38:
Not in scope: type constructor or class ‘Gen’
Failed to install asn1-encoding-0.9.0
cabal: Error: some packages failed to install:
asn1-encoding-0.9.0 failed during the building phase. The exception was:
ExitFailure 1
At some point, newer types for time representation have been introduced into ASN.1.
I don't know in which revision they've been introduced, but at least the latest revision of the X.680 ITU standard includes the ISO 8601 conformant types Time, Date, Date-Time, Interval, Rec-Interval
, which, according to the OSS tutorial should be preferred over UTCTime
or GeneralizedTime
.
Support for representing, encoding and decoding these types would be neat.
I fail to see clear way to encode/decode data with application or context tag.
Only way to do this seem to be by encoding ASN1 with universal tag, parse result with parseBS, swap header and encode again. Similarly in other direction. Could you give some hint how to do this better?
Running the following program
import qualified Data.ByteString as B
import Data.ASN1.Encoding
import Data.ASN1.BinaryEncoding
main = do
f <- B.readFile "data.txt"
print $ decodeASN1Repr' DER f
on the attached file (data.txt) produces a segmentation fault on GHC 7.8.2 and 8.0.1 on Windows.
Currently asn1-data provides enumerator interface for accessing (de)serialization functions. Providing high level easy to use interface is good, except there is more than one implementation of iteratee approach: iteratee, enumerator, conduit, original implementation by Oleg and some toy versions like iterIO or pipes.
It makes no sense to use several implementations of the same concept in your application so asn1-data can be used only by those who use enumerator (or don't mind getting their hands dirty with gory details of library internals or iteratee rishathra).
It would be great do split this package into two different parts - one for providing low level details for those who don't use enumerator and one for providing high level interface for whose who do.
A complete log file of the test run is at http://hydra.nixos.org/build/16373034/nixlog/1/raw.
The latest version and revision of asn1-encoding
requires bytestring < 0.11
. It would be nice if this constraint could be relaxed.
I am using this library for my project, but before getting started with proper usage of it I had to spend several days of experimenting with functions, matching different type signatures to each other, and writing a lot of prototyping code before I figured out how to actually use it.
Most function Haddock annotations provide not much more information than the type signature alone. They are a good start but can use much more verbosity, argument annotations and detailed information.
I am relatively new to Haskell, so not being familiar with concepts like parser combinators definitely increased the additional effort for me.
But what's missing most dearly is a tutorial/ getting started overview of the different parts of the libraries, their use cases, and some basic example code.
The latter can also be useful for adding tests to asn1-parse and asn1-types.
As I'm just starting to wrap my head around this, I might contribute some documentation at a later point of time.
Data.ASN1.Encoding.decodeASN1Repr failed while processing a certificate issued by "SAP Trust Community".
The error is _'TypeDecodingFailed "time format invalid for TimeUTC : cannot convert string 0007181000Z_
I suspected it to be same as -
[https://github.com/elm-lang/elm-reactor/issues/98]
With Yesod now ported to Conduit, hs-asn1-data has become one of the only packages in the Yesod dependencies that still depends on enumerator and attoparsec-enumerator. Please consider switching to conduit and attoparsec-conduit.
Maybe there is valid reason to not expose getEnumerated.
Hey,
Thanks for maintaining these ASN.1 libs! If I'm not mistaken, the UTF-8 encoding implementation in https://hackage.haskell.org/package/asn1-types-0.3.3/docs/src/Data.ASN1.Types.String.html#local-6989586621679034993 is broken due to the last byte always being set null — 0xc0 & 0x3f = 0
. I presume you meant to bitwise-or it with 0x80 instead to have the first two bits be 10
.
It's easy to reproduce with my surname: asn1CharacterToString ("Möll" :: ASN1CharacterString)
Just "M*** Exception: continuation bytes invalid
CallStack (from HasCallStack):
error, called at ./Data/ASN1/Types/String.hs:109:34 in asn1-types-0.3.3-681f1a1bec95ecc832958e8497db7f3872872f9a24538c763fb4706942450148:Data.ASN1.Types.String
Cheers
Hi Vincent,
Can you bump the allowed versions of attoparsec to include 0.9? I can't update http-enumerator without it.
Thanks,
Michael
Here is a script:
#!/usr/bin/env stack
-- stack -v runghc --package connection --package http-client --package http-client-tls --package tls --package data-default
{-# LANGUAGE OverloadedStrings #-}
{-
reproduce memory leak:
$ stack ghc -- -O0 tls-memleak.hs -prof -fprof-auto -rtsopts
doesn't finish:
$ ./tls-memleak +RTS -K1K -p -hy
finishes:
$ ./tls-memleak +RTS -K100K -p -hy
-}
import qualified Network.Connection as NC
import qualified Network.HTTP.Client as Http
import qualified Network.HTTP.Client.TLS as Http
import qualified Network.TLS as TLS
import qualified Network.TLS.Extra as TLS (ciphersuite_all)
import qualified System.X509 as TLS
import Data.Default
import Network.HTTP.Client
import Network.HTTP.Client.TLS
import Prelude
main :: IO ()
main = do
let (Just req) = parseUrl "https://spreadsheets.google.com"
-- @TlsExceptionHostPort (HandshakeFailed Error_EOF) "spreadsheets.google.com" 443@
-- mgr <- newManager tlsManagerSettings
mgr <- mkHttpManager True
res <- httpLbs req mgr
print (show (responseStatus res))
-- | Create an HTTP 'Manager' for running a 'Test'
mkHttpManager :: Bool -- ^ validate ssl
-> IO Manager
mkHttpManager validateSsl = do
scs <- TLS.getSystemCertificateStore
let tlsSettings = NC.TLSSettings (cp scs)
mngrCfg = Http.mkManagerSettings tlsSettings Nothing
Http.newManager mngrCfg
where
cp scs = (TLS.defaultParamsClient "" "") {
TLS.clientSupported = def {
TLS.supportedCiphers = TLS.ciphersuite_all
, TLS.supportedHashSignatures = hashSignatures
-- , TLS.supportedVersions [TLS10, TLS11, TLS12]
}
, TLS.clientShared = def {
TLS.sharedCAStore = scs
, TLS.sharedValidationCache = validationCache
}
}
hashSignatures =
[ (TLS.HashSHA512, TLS.SignatureRSA)
, (TLS.HashSHA384, TLS.SignatureRSA)
, (TLS.HashSHA256, TLS.SignatureRSA)
, (TLS.HashSHA224, TLS.SignatureRSA)
, (TLS.HashSHA1, TLS.SignatureRSA)
, (TLS.HashSHA1, TLS.SignatureDSS)
-- , (TLS.HashSHA512, TLS.SignatureECDSA) -- "bad SignatureECDSA for ecdhparams"
-- , (TLS.HashSHA384, TLS.SignatureECDSA) -- "bad SignatureECDSA for ecdhparams"
, (TLS.HashSHA256, TLS.SignatureECDSA)
, (TLS.HashSHA224, TLS.SignatureECDSA)
, (TLS.HashSHA1, TLS.SignatureECDSA)
]
validationCache =
if not validateSsl then
TLS.ValidationCache
(\_ _ _ -> return TLS.ValidationCachePass)
(\_ _ _ -> return ())
else
def
NOTE: a lot of the code in the script is intended to workaround tls ECDSA
configuration by using a custom function to create the http manager:
When I run this script,
$ stack ghc -- -O0 tls-memleak.hs -prof -fprof-auto -rtsopts
$ ./tls-memleak +RTS -K1K -xc -p -hc
But running w/ this command works as expected:
$ ./tls-memleak +RTS -K100K -xc -p -hc
I realize this is due to limiting stack size to 1K, but there is some
documentation which suggests this is a good way to identify space
leaks
(http://neilmitchell.blogspot.co.uk/2015/09/detecting-space-leaks.html).
Here are some heap profile graphics:
Do you think this is something work investigating? I'll be happy to provide more info if needed.
It looks like this library rolls its own parser under the hood in the internal Data.ASN1.Get
module that looks the same as the parser in the cereal
library. Would it be possible to reuse cereal
and export your parser so that this can be embedded within a larger parser?
Currently, asn1-data and asn1-encoding don't compile, since the define fail
for a Monad
; files affected are:
data/Data/ASN1/Get.hs:108
encoding/Data/ASN1/Get.hs:106
The intuitive (to me) way to fix this would be using a conditional compilation similar to what you've done already in parse/Data/ASN1/Parse.hs:63:
#if MIN_VERSION_base(4,9,0)
instance MonadFail ParseASN1 where
fail = throwParseError
#endif
by checking against GHC 8.8.1 base version, so that definitions remain unchanged for earlier base versions. In case you agree, I could prepare a PR.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.