typelevel / scalacheck

Property-based testing for Scala

ScalaCheck

ScalaCheck is a library written in Scala and used for automated property-based testing of Scala or Java programs. ScalaCheck was originally inspired by the Haskell library QuickCheck, but has also ventured into its own.

ScalaCheck has no external dependencies other than the Scala runtime, and works great with SBT, the Scala build tool. It is also fully integrated in the test frameworks ScalaTest and specs2. You can of course also use ScalaCheck completely standalone, with its built-in test runner.

ScalaCheck is used by several prominent Scala projects, for example the Scala compiler and the Akka concurrency framework.

For more information and downloads, please visit http://www.scalacheck.org

scalacheck's Issues

Prop.within method can stack overflow.

I've got a fix for this one, the attempt method at:
https://github.com/rickynils/scalacheck/blob/master/src/main/scala/org/scalacheck/Prop.scala#L748
just needs "@tailrec private" added to stack bust.

Add an applicative notation for generators

The for/yield notation can be used with generators because they are monads, but I sometimes wish a more concise notation when just applicatives are needed (because generators are independent):

Contrast:

case class Time(hour: Int, minute: Int)

for {
  hour <- Gen.choose(0, 23)
  minute <- Gen.choose(0, 59)
} yield Time(hour, minute)

with (using the scalaz syntax)

(Gen.choose(0, 23) |@| Gen.choose(0, 59))(Time)

Move to Sonatype repo?

It seems like ScalaCheck is not availale in the Sonatype repo.

Would it possible to have a SBT repo string somewhere in the documentation, not only the Maven XML?

arbitrary[BigDecimal] causes ArithmeticException: Overflow

Hi,
I had some trouble with sporadic failures while generating Arbitrary BigDecimals. Here is some code that shows the problem.
Using version "org.scala-tools.testing" %% "scalacheck" % "1.9"

import org.scalacheck.{Gen, Arbitrary}
import Arbitrary.arbitrary

def testBigDecimalGen = { val x = genPremium ; (0 to 1000) foreach ( x => println(x + " => " + arbitrary[BigDecimal].sample)) }

at some point (usually before 100 iterations) the error manifests here's a sample :-

....
33 => Some(1.054685299389887E+1528400977)
34 => Some(-5.435685773778648E+33)
35 => Some(-3.572964852669853866365433378033566220288E-1426971473)
36 => Some(1.3546827679130451968E+2079363306)
[info] PremiumSpec
[info]
[error] ! Test the BigDecimal Generator
[error] ArithmeticException: Overflow (BigDecimal.java:3741)
[error] org.scalacheck.Arbitrary$$anonfun$1$$anonfun$apply$5$$anonfun$apply$6.apply(Arbitrary.scala:183)
[error] org.scalacheck.Arbitrary$$anonfun$1$$anonfun$apply$5$$anonfun$apply$6.apply(Arbitrary.scala:182)
[error] org.scalacheck.Gen$$anonfun$map$1.apply(Gen.scala:90)
[error] org.scalacheck.Gen$$anonfun$map$1.apply(Gen.scala:90)
[error] org.scalacheck.Gen$$anon$3.apply(Gen.scala:231)
[error] org.scalacheck.Gen$$anonfun$flatMap$1$$anonfun$apply$21.apply(Gen.scala:109)
[error] org.scalacheck.Gen$$anonfun$flatMap$1$$anonfun$apply$21.apply(Gen.scala:108)
[error] org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:108)
[error] org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:107)
[error] org.scalacheck.Gen$$anon$3.apply(Gen.scala:231)
[error] org.scalacheck.Gen$$anonfun$flatMap$1$$anonfun$apply$21.apply(Gen.scala:109)
[error] org.scalacheck.Gen$$anonfun$flatMap$1$$anonfun$apply$21.apply(Gen.scala:108)
[error] org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:108)
[error] org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:107)
[error] org.scalacheck.Gen$$anon$3.apply(Gen.scala:231)
[error] org.scalacheck.Gen$class.sample(Gen.scala:187)
[error] org.scalacheck.Gen$$anon$3.sample(Gen.scala:230)
[error] models.PremiumTests$premiums$$anonfun$testBigDecimalGen$1.apply$mcVI$sp(Quote2Tests.scala:61)
[error] models.PremiumTests$premiums$.testBigDecimalGen(Quote2Tests.scala:61)
[error] models.PremiumTests$class.bigDecimalSpec(Quote2Tests.scala:52)
[error] models.PremiumSpec.bigDecimalSpec(Quote2Spec.scala:21)
[error] models.PremiumSpec$$anonfun$is$2$$anonfun$apply$15.apply(Quote2Spec.scala:22)
[error] models.PremiumSpec$$anonfun$is$2$$anonfun$apply$15.apply(Quote2Spec.scala:22)
[info]
[info] Total for specification PremiumSpec
[info] Finished in 45 ms
[info] 1 example, 0 failure, 1 error
[info]

hope this helps.
Cheers
Karl

Add properties for untested methods in Prop

There's a lot of untested things in org.scalacheck.Prop. Some examples are Prop.sizedProp, Prop.someFailing, Prop.noneFailing, but there's many more. There should be properties in test/scala/org/scalacheck/PropSpecification.scala for essentially every public method in org.scalacheck.Prop.

User-Guide contains strange hex number

The section Test Execution contains some seemingly random hex numbers:

"The check method looks like this: 43bbc1d6b666fa6482659729769855fd"

I hope it is not a joke I don't understand.

Properties that check if exceptions are thrown don't compile with Scala 2.9.1

Scala 2.9.x doesn't seem to be able to resolve the throws method correctly. Given this test, which uses the exact same code as ScalaCheck 1.10.1's PropSpecification:

package com.simple.test

import org.junit.runner.RunWith
import org.scalacheck.Properties
import org.scalacheck.Prop._
import org.scalacheck.contrib.ScalaCheckJUnitPropertiesRunner

@RunWith(classOf[ScalaCheckJUnitPropertiesRunner])
class BareSpec extends Properties("Failure case") {
  property("throws") = forAll { n: Int =>
    if (n == 0) throws(classOf[ArithmeticException])(1/0)
    else true
  }
}

… The project fails to compile, with:

[ERROR] /Users/ieure/Projects/simple/test/src/test/scala/com/simple/test/BareSpec.scala:12: not enough arguments for method throws: (x: => Any, c: Class[T])org.scalacheck.Prop.
Unspecified value parameter c.
[ERROR]     if (n == 0) throws(classOf[ArithmeticException])(1/0)
[ERROR]                       ^
[ERROR] one error found
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.1.0:testCompile (test-scalac) on project buckingham: Execution test-scalac of goal net.alchim31.maven:scala-maven-plugin:3.1.0:testCompile failed. CompileFailed -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException

The older (deprecated) style still works, even though it seems to use the new multi-arg-list version internally:

  property("throws") = forAll { n: Int => (n == 0) ==>
    throws({ n/0 }, classOf[ArithmeticException])
  }

Move test callbacks out of Test.check

Today, the test runner is a bit mixed up with the property evaluator. It should be possible to make a clean Test.check method that just takes a property and returns a test result. The test runner could be implemented separately as a special property that wraps another property and intercepts each evaluation. This would reduce the complexity in Test.check which would help when implementing new features (such as a timeout check, as mentioned in #5 and #6).

NoSuchMethodError: scala.runtime.IntRef.zero()

When running the example from the readme,either on ScalaCheck 1.10.1 or 1.10.11, with Scala 2.10.2, Java 7.40, there is a crash with the following message:

Exception in thread "main" java.lang.NoSuchMethodError: scala.runtime.IntRef.zero()Lscala/runtime/IntRef;
    at org.scalacheck.Gen$.frequency(Gen.scala)
    at org.scalacheck.Arbitrary$$anonfun$arbChar$1.apply(Arbitrary.scala:114)
    at org.scalacheck.Arbitrary$$anonfun$arbChar$1.apply(Arbitrary.scala:114)
    at org.scalacheck.Arbitrary$$anon$2.arbitrary$lzycompute(Arbitrary.scala:65)
    at org.scalacheck.Arbitrary$$anon$2.arbitrary(Arbitrary.scala:65)
    at org.scalacheck.Arbitrary$.arbitrary(Arbitrary.scala:69)
    at org.scalacheck.Arbitrary$$anonfun$arbContainer$1.apply(Arbitrary.scala:299)
    at org.scalacheck.Arbitrary$$anonfun$arbContainer$1.apply(Arbitrary.scala:299)
    at org.scalacheck.Arbitrary$$anon$2.arbitrary$lzycompute(Arbitrary.scala:65)
    at org.scalacheck.Arbitrary$$anon$2.arbitrary(Arbitrary.scala:65)
    at org.scalacheck.Arbitrary$.arbitrary(Arbitrary.scala:69)
    at org.scalacheck.Arbitrary$$anonfun$arbString$1.apply(Arbitrary.scala:137)
    at org.scalacheck.Arbitrary$$anonfun$arbString$1.apply(Arbitrary.scala:137)
    at org.scalacheck.Arbitrary$$anon$2.arbitrary$lzycompute(Arbitrary.scala:65)
    at org.scalacheck.Arbitrary$$anon$2.arbitrary(Arbitrary.scala:65)
    at org.scalacheck.Arbitrary$.arbitrary(Arbitrary.scala:69)
    at org.scalacheck.Prop$.forAll(Prop.scala:726)
    at org.scalacheck.Prop$.forAll(Prop.scala:734)
    at scalacheck.StringSpecification$.<init>(StringSpecification.scala:8)
    at scalacheck.StringSpecification$.<clinit>(StringSpecification.scala)
    at scalacheck.StringSpecification.main(StringSpecification.scala)

support all standard Collections using CanBuildFrom

the following is a Builder impl that supports all standard collection types using the CanBuildFrom machinery:

implicit def buildableCanBuildFrom[T, C[_]](implicit c: CanBuildFrom[C[_], T, C[T]]) =
  new Buildable[T, C] {
    def builder = c.apply
  }

I'll try and add a PR

Add "generator invariants"

Transferred from http://code.google.com/p/scalacheck/issues/detail?id=85

Make it possible to attach preconditions/invariants to generators so you don't have to add a precondition to the property also.

See http://groups.google.com/group/scalacheck/msg/b563f24c81b22291

Wrong path in command file for sbt

In sbt.cmd:
java %SBT_OPTS% %PROXY_OPTS% -Xmx1024M -jar "%SCRIPT_DIR%sbt-launch-0.7.4.jar" %*
should be
java %SBT_OPTS% %PROXY_OPTS% -Xmx1024M -jar "%SCRIPT_DIR%/tools/sbt-launch-0.7.4.jar" %*

Delete me

Oops, this isn't scalacheck related. Sorry...

1.10-SNAPSHOT incompatible with specs2 1.9.

I'm not entirely sure what has changed, but it would appear whatever it is that has changed has caused it to become incompatible with the current specs2 integration:
[info] JSONIntSpecification
[info]
[info] intValue
[error] ! Fragment evaluation error
[error] ThrowableException: org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V (FutureTask.java:166)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$7$$anonfun$apply$8$$anonfun$apply$11.apply(JSONIntSpecification.scala:17)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$7$$anonfun$apply$8$$anonfun$apply$11.apply(JSONIntSpecification.scala:17)
[error] org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$7$$anonfun$apply$8$$anonfun$apply$11.apply(JSONIntSpecification.scala:17)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$7$$anonfun$apply$8$$anonfun$apply$11.apply(JSONIntSpecification.scala:17)
[error] ! Fragment evaluation error
[error] ThrowableException: org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V (FutureTask.java:166)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$7$$anonfun$apply$14.apply(JSONIntSpecification.scala:22)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$7$$anonfun$apply$14.apply(JSONIntSpecification.scala:22)
[error] org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$7$$anonfun$apply$14.apply(JSONIntSpecification.scala:22)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$7$$anonfun$apply$14.apply(JSONIntSpecification.scala:22)
[error] ! Fragment evaluation error
[error] ThrowableException: org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V (FutureTask.java:166)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$17.apply(JSONIntSpecification.scala:28)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$17.apply(JSONIntSpecification.scala:28)
[error] org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$17.apply(JSONIntSpecification.scala:28)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$6$$anonfun$apply$17.apply(JSONIntSpecification.scala:28)
[error] ! Fragment evaluation error
[error] ThrowableException: org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V (FutureTask.java:166)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$19.apply(JSONIntSpecification.scala:34)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$19.apply(JSONIntSpecification.scala:34)
[error] org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$19.apply(JSONIntSpecification.scala:34)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$5$$anonfun$apply$19.apply(JSONIntSpecification.scala:34)
[error] ! Fragment evaluation error
[error] ThrowableException: org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V (FutureTask.java:166)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$21.apply(JSONIntSpecification.scala:39)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$21.apply(JSONIntSpecification.scala:39)
[error] org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$21.apply(JSONIntSpecification.scala:39)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$intValueSpec$1$$anonfun$apply$21.apply(JSONIntSpecification.scala:39)
[info] longValue
[error] ! Fragment evaluation error
[error] ThrowableException: org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V (FutureTask.java:166)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$22$$anonfun$apply$23$$anonfun$apply$26.apply(JSONIntSpecification.scala:46)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$22$$anonfun$apply$23$$anonfun$apply$26.apply(JSONIntSpecification.scala:46)
[error] org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$22$$anonfun$apply$23$$anonfun$apply$26.apply(JSONIntSpecification.scala:46)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$22$$anonfun$apply$23$$anonfun$apply$26.apply(JSONIntSpecification.scala:46)
[error] ! Fragment evaluation error
[error] ThrowableException: org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V (FutureTask.java:166)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$22$$anonfun$apply$28.apply(JSONIntSpecification.scala:52)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$22$$anonfun$apply$28.apply(JSONIntSpecification.scala:52)
[error] org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$22$$anonfun$apply$28.apply(JSONIntSpecification.scala:52)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$22$$anonfun$apply$28.apply(JSONIntSpecification.scala:52)
[error] ! Fragment evaluation error
[error] ThrowableException: org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V (FutureTask.java:166)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$30.apply(JSONIntSpecification.scala:57)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$30.apply(JSONIntSpecification.scala:57)
[error] org.scalacheck.Test$Params.(IIIILjava/util/Random;ILorg/scalacheck/Test$TestCallback;)V
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProperty(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification.checkProp(JSONIntSpecification.scala:14)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$30.apply(JSONIntSpecification.scala:57)
[error] com.github.seanparsons.jsonar.JSONIntSpecification$$anonfun$longValueSpec$1$$anonfun$apply$30.apply(JSONIntSpecification.scala:57)
[info]
[info] Total for specification JSONIntSpecification
[info] Finished in 731 ms
[info] 8 examples, 0 failure, 8 errors

Easy way to test it is to grab the latest code from here: https://github.com/seanparsons/jsonar I've got master wired up to use 1.10-SNAPSHOT and it explodes anywhere that uses scalacheck.

Arbitrary instance for Option[A] does not have good coverage.

Since the Arbitrary instance for Option[A] will only generate None when size is zero, it's easy to write passing tests that should fail:

scala> import org.scalacheck.Prop.forAll
import org.scalacheck.Prop.forAll

scala> (forAll { (x: Option[Int], y: Option[Int]) => x.isDefined == y.isDefined }).check
+ OK, passed 100 tests.

Create Ant task for ScalaCheck

Transferred from http://code.google.com/p/scalacheck/issues/detail?id=23

Allow `Gen.Params` to be subclassed

For some of the more involved types I'm generating, it would be very nice to subclass Gen.Params to control the generator at a fine-grained level. For example, I need to test a case class representing the creation of a bank account, and I need to generate both valid routing/account numbers and invalid / unset ones to ensure that bad requests are detected and handled. Using suchThat with more than one field on a case class nearly always results in generator exhaustion, so I need to explicitly generate valid/invalid types.

Currently, I'm doing this with a function like:

def accountGenerator(genRoutingAccount: Gen[(Option[String], Option[String]]) = for {(routing, account) <- genRoutingAccount … } yield CreateAccount(…)}

But it seems more idiomatic to use Gen.Params to control things like this. This is not currently possible because, while I can subclass Gen.Params (with a normal class, not a case class):

  class CustomParams extends Gen.Params

  val tg: Gen[String] = Gen({p: CustomParams => Option("x")})

It fails with:

ERROR: type mismatch;  found   : TestGenerator.this.CustomParams => Option[java.lang.String]  required: org.scalacheck.Gen.Params => Option[String] : line 221

This is because the type bounds of Gen.apply don't accept subclasses of Gen.Params.

Container gen composition

This is more of a request for documentation or help rather than a feature or bug.

I have a set of generators that generate akka.util.ByteStrings. Some of the generators generate containersOfN or containersOf1, some generate data of fixed lengths. Some larger generators are composed of both the containersOf and fixed length generators, and then flattened:
val bigContainer = for {
len3byteString <- Len3.gen
len4byteString <- Len4.gen
list1 <- listOfN(len3byteString, len4byteString)
list2 <- listOfN(len3byteString, len4byteString, list1)
list3 <- listOfN(len3byteString, len4byteString, list1, list2)
} yield ByteString(list3.flatten : _*)

Predictably, the size of these generators using flatten explodes at like, O(n^2). What I'd like to do is place an upper bounds on the size of the output of the resulting generator bigContainer, so that the length of the tests doesn't increase without bounds as the complexity of the containers(and container containers) increases. Is there a pattern or combinator to compose containers(that are later flattened) to control the resulting sizes? Relevant link would be :
http://etorreborre.blogspot.ca/2011/02/scalacheck-generator-for-json.html

Add zip operators on generators

Something like this for a tuples up to 22 members:

 implicit class ZippedGen[T1](g1: Gen[T1]) {
    def zip[T2](g2: Gen[T2]) = for {
      t1 <- g1
      t2 <- g2
    } yield (t1, t2)
  }

So that we can write:

val hour: Gen[Int]   = Gen.choose(0, 23)
val minute: Gen[Int] = Gen.choose(0, 59)
val time: Gen[(Int, Int)] = hour zip minute

Show required import in User Guide

In the section "Labeling Properties", show what feature-specific imports are necessary and preferred - probably just

import org.scalacheck.Prop._

The compiler error that you get otherwise

 value :| is not a member of Boolean 

is hard to diagnose.

Shrinking two generators sometimes presents strange ORIGINAL values.

scala> val p = forAll(choose(1000,2000), choose(1,2000))(_ > _)
p: org.scalacheck.Prop = Prop

scala> p.check
! Falsified after 0 passed tests.
> ARG_0: 0
> ARG_0_ORIGINAL: 1082
> ARG_1: 0
> ARG_1_ORIGINAL: 832

Testing should stop once Gen is exhausted

Transferred from http://code.google.com/p/scalacheck/issues/detail?id=84

object Simple extends Properties("easily exhausted") {

  val oneToFive = Gen.oneOf(Range(1,6).toList)

  property("for i in [1..5], i < 6") = forAll(oneToFive) { i: Int =>
    i < 6
  }
}
+ .qualac.lex.BadProgram: OK, passed 100 tests.

It would be really nice if ScalaCheck would stop after 5 tests since there are only 5 possible tests. Maybe Test.Params could have an option that lets the user guarantee referential transparency, in which case the runner would only go to the size of the Gen space.

ScalaCheck for Scala 2.9.2

I would like to publish a specs2 version for Scala 2.9.2, can you please publish a ScalaCheck version for the same?

Thanks.

Add max testing duration

Transferred from http://code.google.com/p/scalacheck/issues/detail?id=83
#82 requests a timeout for any particular test. It would also be nice if Test.Params had an option for max duration of the entire testing run. For example, instead of specifying minSuccessfulTests and maxDiscardedTests, I'd like to just specify that testing go for 8 hours. There are any number of ways this feature could interact with the other parameters, as discussed on the mailing list. All would be fine with me!

Resolvers as specified in the readme can potentially trip up the unwary.

This:
resolvers ++= Seq(
"snapshots" at "http://oss.sonatype.org/content/repositories/snapshots",
"releases" at "http://oss.sonatype.org/content/repositories/releases"
)
should really be:
resolvers ++= Seq(
"Sonatype Snapshots" at "http://oss.sonatype.org/content/repositories/snapshots",
"Sonatype Releases" at "http://oss.sonatype.org/content/repositories/releases"
)

As the default examples for using publishTo in SBT also use "snapshots" and "releases" and the above can overwrite those.

scalacheck_2.9.3

Please build and release scalacheck for 2.9.3

1.9 for Scala 2.9.0-1

Hi Ricky,

Could you please publish an official version for Scala 2.9.0-1?

Thanks.

Update to SBT 0.10

Remove mutable random number generator in Gen

Gen.Parameter includes rng which is an instance of java.util.Random. This instance is used in Gen.choose, by calls to rng.nextLong and rng.nextDouble. These calls mutate the rng instance, by updating its seed. It would be nice to switch over to an immutable rng, since that would allow control over the seed (which would enable repeatable tests in ScalaCheck). So, instead of java.util.Random we will have seed: Long and rng: Long => Long in Gen.Parameters.

The change is not very big, we just need to make sure that the generators update the seed correctly. I have started to implement this, but I don't want to push the change to 1.11.0 since I've already done considerable changes there.

bad symbolic reference. A signature in ScalaCheckParameters.class refers to term Pretty

In scalacheck 1.11.0 I have the following error (seems like a LinkageError in Java)

[error] bad symbolic reference. A signature in ScalaCheckParameters.class refers to term Pretty
[error] in package org.scalacheck which is not available.
[error] It may be completely missing from the current classpath, or the version on
[error] the classpath might be incompatible with the version used when compiling ScalaCheckParameters.class.
[error] bad symbolic reference. A signature in ScalaCheckParameters.class refers to type Params
[error] in value org.scalacheck.Pretty which is not available.
[error] It may be completely missing from the current classpath, or the version on
[error] the classpath might be incompatible with the version used when compiling ScalaCheckParameters.class.
[error] two errors found
[error] (test:compile) Compilation failed
[error] Total time: 1 s, completed 5 nov. 2013 17:32:20

my environment is

dependencies :
"org.specs2"          %%  "specs2"        % "2.2.3" % "test",
"org.scalacheck" %% "scalacheck" % "1.11.0" % "test"

scalaVersion  := "2.10.3"
sbt.version=0.13.0

Replace implicit conversions with typeclasses

There are quite a few places in ScalaCheck where implicit conversions are expected:

def forAll[A1,P] (
    f: A1 => P)(implicit
    p: P => Prop,
    a1: Arbitrary[A1], s1: Shrink[A1], pp1: A1 => Pretty
  ): Prop = forAllShrink(arbitrary[A1],shrink[A1])(f andThen p)

The trouble with this formulation is that, when using the p conversion, there is no way to catch exceptions if the parameter of type P throws any. My proposal is to use a typeclass instead:

trait ToProp[P] {
  def toProp(p: =>P): Prop
}

def forAll[A1,P] (
    f: A1 => P)(implicit
    p: ToProp[P],
    a1: Arbitrary[A1], s1: Shrink[A1], pp1: A1 => Pretty
  ): Prop = forAllShrink(arbitrary[A1],shrink[A1])(f andThen p)

The same thing can be done with the Pretty conversion.

Alternatively you could require an implicit conversion of the form (=>P) => Prop and that should work.

Mpi-calls deadlock inside "Property"s

Simple code below deadlocks using MPJ-Express. I have used MPJ-express extensively during the last 3 months, and this code has never failed, however, it leads to deadlock inside the property. It runs fine without the property.

Basically, 4 processes are run with this scala-code.

object CollectionCheck {

def main(args: Array[String]) {

  MPI.Init(args)
  val ag = forAll((n: Int) => {
    System.out.println(MPI.COMM_WORLD.Rank() + "/" + MPI.COMM_WORLD.Size() + ": BEFORE ")
    MPI.COMM_WORLD.Barrier()
    System.out.println("AFTER")
    1 == 1
  }).check

}

}

The output from this code is as follows:

MPJ Express (0.38) is started in the multicore configuration
0/4: BEFORE
2/4: BEFORE
1/4: BEFORE
3/4: BEFORE

and then the program hangs. Any suggestions?

Exception raised on argument generation. > Exception: java.lang.NullPointerException: null (Gen.scala:291)

I'm currently seeing this exception that is being thrown from Gen.scala. I haven't been able to find a base case for it yet. Any ideas what could be causing this? I'm using scalacheck 1.10.0.

Exception raised on argument generation.
> Exception: java.lang.NullPointerException: null
java.lang.Exception: Exception raised on argument generation.
> Exception: java.lang.NullPointerException: null
    at org.scalacheck.Gen$$anonfun$frequency$1.apply(Gen.scala:292)
    at org.scalacheck.Gen$$anonfun$frequency$1.apply(Gen.scala:291)
    at org.scalacheck.Gen$$anonfun$flatMap$1$$anonfun$apply$21.apply(Gen.scala:115)
    at org.scalacheck.Gen$$anonfun$flatMap$1$$anonfun$apply$21.apply(Gen.scala:114)
    at org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:114)
    at org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:113)
    at org.scalacheck.Gen$$anon$3.apply(Gen.scala:225)
    at org.scalacheck.Gen$$anonfun$sequence$1.apply(Gen.scala:241)
    at org.scalacheck.Gen$$anonfun$sequence$1.apply(Gen.scala:237)
    at org.scalacheck.Gen$$anon$3.apply(Gen.scala:225)
    at org.scalacheck.Gen$$anonfun$filter$1.apply(Gen.scala:119)
    at org.scalacheck.Gen$$anonfun$filter$1.apply(Gen.scala:118)
    at org.scalacheck.Gen$$anon$3.apply(Gen.scala:225)
    at org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:114)
    at org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:113)
    at org.scalacheck.Gen$$anon$3.apply(Gen.scala:225)
    at org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:114)
    at org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:113)
    at org.scalacheck.Gen$$anon$3.apply(Gen.scala:225)
    at org.scalacheck.Gen$$anonfun$flatMap$1$$anonfun$apply$21.apply(Gen.scala:115)
    at org.scalacheck.Gen$$anonfun$flatMap$1$$anonfun$apply$21.apply(Gen.scala:114)
    at org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:114)
    at org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:113)
    at org.scalacheck.Gen$$anon$3.apply(Gen.scala:225)
    at org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:114)
    at org.scalacheck.Gen$$anonfun$flatMap$1.apply(Gen.scala:113)
    at org.scalacheck.Gen$$anon$3.apply(Gen.scala:225)
    at org.scalacheck.Prop$$anonfun$forAllShrink$1.apply(Prop.scala:566)
    at org.scalacheck.Prop$$anonfun$forAllShrink$1.apply(Prop.scala:542)
    at org.scalacheck.Prop$$anon$1.apply(Prop.scala:283)
    at org.scalacheck.Test$$anonfun$worker$1$1$$anonfun$apply$2.apply(Test.scala:300)
    at org.scalacheck.Test$$anonfun$worker$1$1$$anonfun$apply$2.apply(Test.scala:300)
    at org.scalacheck.Test$.org$scalacheck$Test$$secure(Test.scala:203)
    at org.scalacheck.Test$$anonfun$worker$1$1.apply(Test.scala:300)
    at org.scalacheck.Test$$anonfun$worker$1$1.apply(Test.scala:291)

Build for Scala 2.10.0 RC2

Scala 2.10.0 RC2 is staged so it would be great to have a build for it:

https://groups.google.com/d/topic/scala-internals/R9JzcvPG7h8/discussion

Thanks!

conditional property doc example

After going through the very nice ScalaCheck tutorial and using ScalaCheck successfully for a Coursera homework assignment I am puzzled about the conditional property example in the tutorial. The example works:

val propMakeList = forAll { n: Int =>
  (n >= 0 && n < 10000) ==> (List.make(n, "").length == n)
}

What I thought was a similar example does not work:

val rangeLengthSmallIntCP = forAll { x: Int =>
  (x > 0 && x < 1000) ==> ((1 to x).size == x)
}
rangeLengthSmallCP.check
! Gave up after only 15 passed tests. 86 tests were discarded.

Now I understand that random sampling of the 1e9 Int space is has a very small chance of choosing in the the range (1 to 1000), but I would think that would cause the tutorial example to also give up. My example fails (gives up) even if I adjust the bounds to 1e4 or 1e5.

I understand I can write:

val rangeLengthSmallIntGen = forAll(Gen.choose(1, 1000)) { x: Int =>
  (1 to x).size == x
}

and that's a perfectly clear expression of the intent.

[1] Why does my example fail when the tutorial example works?

[2] Is the forAll(Gen.choose()) example perhaps a better best practice example to use in the tutorial for limiting tests to small ints?

Thanks for your time and for creating a really nifty package.

java.lang.NoClassDefFoundError: scala/Right when using Scala 2.10 rc2

Hello,

After switching to Scala 2.10 (due to need for TypeTags) I get the following error:

java.lang.NoClassDefFoundError: scala/Right
at org.scalacheck.Prop$.forAllShrink(Prop.scala:542)....

Can I make this work with 2.10?

ambiguous Buildable[T, Set] error

there are two Buildable[T, Set] instances in the Buildable companion:

implicit def buildableImmutableSet[T] = new Buildable[T,Set] {
  def builder = new mutable.SetBuilder(Set.empty[T])
}

and
implicit def buildableSet[T] = new Buildable[T,Set] {
def builder = Set.newBuilder
}

leading to ambiguous implicits error when trying to summon one

Link to public ScalaDoc?

Can you please include a link to the ScalaDoc in the README? I can't find current documentation anywhere.

Change Test.Params.maxDiscardedTests to maxDiscardRatio

Instead of specifying an absolute number of the maximum discarded tests (the generator fails to generate a value, or the precondition is not fulfilled) allowed per property, it would be more convenient to define the maximum allowed ratio between discarded and passed property evaluations. This would also interact better with issue #6.

Gen behavior

As I am going through the tutorial of ScalaCheck , I decided to modify one of the examples , from

object mytests {
 def main(args: Array[String]): Unit = {
 val smallInteger = Gen.choose(0,100)
val propSmallInteger = Prop.forAll(smallInteger)(n => n >= 0 && n <= 100)
    propSmallInteger.check  }}

to

object mytests {  def main(args: Array[String]): Unit = {
  val smallInteger = Gen.choose(0,100)
val propSmallInteger = Prop.forAll(smallInteger)(n => n >= 0 && n <= 99) //changed 100 to 99
    propSmallInteger.check  }}

And then I ran the program in eclipse , got

 + OK, passed 100 tests.

and then ran it again and got

! Falsified after 99 passed tests.> ARG_0: -1

So my questions are:

• Does the "-1" means that the generator generated -1 event though it has a range of (0,100)
• Why is the result not consistent ? first pass then no pass
  Any idea?

<deleted>

Move User Guide from Google Code to GitHub

Shrinking for choose and chooseNum considers invalid values

For a property like

property("chooseNum") = forAll(chooseNum(0, 10))(n => 0 <= n && n < 10)

Shrinking will settle on -1 instead of 10, and it even considers several out-of-range values during shrinking. From my gut feeling I'd think this is a bug.

java.lang.AbstractMethodError propSqrt.check [scala 2.10.0-m3]

ScalaCheck 1.10.0
Scala version: 2.10.0-M3,

Perhaps this has to do with using a bleeding edge version of Scala, but there's a problem likely with "==>".

First the working version:

scala> val sqrtPlain = forAll { (n: Int) => (Math.sqrt(n*n) == n) }
scala> sqrtPlain.check
! Falsified after 0 passed tests.
> ARG_0: -1

Adding an implication to constrain n. This fails.

scala> val propSqrt = forAll { (n: Int) => (n > 0) ==> (Math.sqrt(n*n) == n) }
propSqrt: org.scalacheck.Prop = Prop

scala> propSqrt.check
! Exception raised on property evaluation.
> ARG_0: 0
> Exception: java.lang.AbstractMethodError: $line48.$read$$iw$$iw$$iw$$iw$$iw$$iw$$anonfun$1.apply(I)Ljava/lang/Object;
scala.Function1$mcLI$sp$$anonfun$andThen$mcLI$sp$1.apply(Function1.scala:55)
scala.Function1$mcLI$sp$$anonfun$andThen$mcLI$sp$1.apply(Function1.scala:55)
org.scalacheck.Prop$$anonfun$forAllShrink$1$$anonfun$4$$anonfun$5.apply(Prop.scala:548)
org.scalacheck.Prop$.secure(Prop.scala:404)
org.scalacheck.Prop$$anonfun$forAllShrink$1$$anonfun$4.apply(Prop.scala:548)

This has to do with the implication, not with calling sqrt, since the following equivalent statement works fine.

scala> val propSqrt = forAll { (n: Int) => n <= 0 || (Math.sqrt(n*n) == n) }    
scala> propSqrt.check
! Falsified after 6 passed tests.
> ARG_0: 46341

Add timeout feature to tests

Transferred from http://code.google.com/p/scalacheck/issues/detail?id=82

Some tests my result in an infinite loop, and this may not be predictable in advance. It would be great if the runner supported an option that kills any test that has run longer than X seconds.

Arbitrary[Int] doesn't respect maxSize (as User Guide implies)

The User Guide says:

See the following implicit Arbitrary definition for integers, that comes from the ScalaCheck implementation.

implicit def arbInt: Arbitrary[Int] = Arbitrary(Gen.sized (s => Gen.choose(-s,s)))

But that isn't the real definition, the actual definition in the source code is:

implicit lazy val arbInt: Arbitrary[Int] = Arbitrary(Gen.chooseNum(Int.MinValue, Int.MaxValue))

And the same for other types like Long and so on.

So either the User Guide should be changed, or the code should be changed. What do you think? Is there some reason not to change the code? Somehow I have the feeling the way the code is what it is intentionally...?

Build for Scala 2.10.0 M7

Hi,

Is there any chance we can get a scalacheck build for 2.10.0 M7? See the following for more details:

https://groups.google.com/d/topic/scala-internals/-jF_L3y5J5s/discussion

Best,
Ismael

ClassCastException with containers (arrays?) of primitive types

I believe d2e6f07 caused a regression, reproduced by the test in adriaanm@c669002 of adriaanm#2

The cast in https://github.com/rickynils/scalacheck/blob/master/src/main/scala/org/scalacheck/Gen.scala#L62 does break stuff once in a while.

 override def sieveCopy(x: Any) = f(x.asInstanceOf[T])

Repro:

scala> import org.scalacheck._
import org.scalacheck._

scala> object P extends Properties("Sieve mustn't CCE") {
     |   property("symmetry") = Prop.forAll { (x: AnyVal, y: AnyVal) => (x == y) == (y == x) }
     |  }
defined object P

scala> P.check

Output:

! Sieve mustn't CCE.symmetry: Exception raised on argument generation.
> Exception: java.lang.ClassCastException: java.lang.Integer cannot be cast
   to java.lang.Character
scala.runtime.BoxesRunTime.unboxToChar(BoxesRunTime.java:93)
org.scalacheck.Gen$Choose$$anon$12$$anonfun$choose$17.apply(Gen.scala:241)
org.scalacheck.Gen$$anon$2.sieveCopy(Gen.scala:62)
org.scalacheck.Gen$$anonfun$frequency$2$$anonfun$apply$12.apply(Gen.scala:3
  41)
org.scalacheck.Gen$$anonfun$frequency$2$$anonfun$apply$12.apply(Gen.scala:3
  41)

Build for Scala 2.10.0-RC3

Scala 2.10.0-RC3 is staged:

https://groups.google.com/d/topic/scala-internals/B-bp7tgOFUE/discussion

Thanks!

Thread context class-loader problem

AFAICS scalacheck creates workers to check properties asynchronously, i.e. they are executed in scala.actors.Futures. These future-blocks are submitted to the single process-wide scala.actors.DaemonScheduler instance. This means that threads are potentially re-used between successive future calls.

In my setup this leads to problems where the wrong class-loader is effective during property execution. I have setup a multi-project sbt project. All projects use scalacheck and as the scala.actors.DaemonScheduler is constant accross all test executions over these projects, a re-used thread's context class-loader is constant as well.

Since org.scalatools.testing.Framework.testRunner gets passed in the correct class-loader from sbt, I propose to use it during worker execution. Here is a patch which passes the class-loader via Test.Params to Test.check and Test.checkProperties and inside future { ... } first calls Thread.currentThread.setContextClassLoader with this class-loader.

diff --git a/src/main/scala/org/scalacheck/ScalaCheckFramework.scala b/src/main/scala/org/scalacheck/ScalaCheckFramework.scala
index 96d6681..0392643 100644
--- a/src/main/scala/org/scalacheck/ScalaCheckFramework.scala
+++ b/src/main/scala/org/scalacheck/ScalaCheckFramework.scala
@@ -69,7 +69,7 @@ class ScalaCheckFramework extends Framework {
       import Test.cmdLineParser.{Success, NoSuccess}
       val prms = Test.cmdLineParser.parseParams(args) match {
         case Success(params, _) =>
-          params copy (testCallback = testCallback)
+          params copy (testCallback = testCallback, customClassLoader = Some(loader))
         // TODO: Maybe handle this a bit better than throwing exception?
         case e: NoSuccess => throw new Exception(e.toString)
       }
diff --git a/src/main/scala/org/scalacheck/Test.scala b/src/main/scala/org/scalacheck/Test.scala
index d0d777c..e339195 100644
--- a/src/main/scala/org/scalacheck/Test.scala
+++ b/src/main/scala/org/scalacheck/Test.scala
@@ -29,7 +29,8 @@ object Test {
     rng: java.util.Random = Gen.Params().rng,
     workers: Int = 1,
     testCallback: TestCallback = new TestCallback {},
-    maxDiscardRatio: Float = 5
+    maxDiscardRatio: Float = 5,
+    customClassLoader: Option[ClassLoader] = None
   )

   /** Test statistics */
@@ -181,6 +182,7 @@ object Test {
     var stop = false

     def worker(workerIdx: Int) = future {
+      params.customClassLoader.map(Thread.currentThread.setContextClassLoader(_))
       var n = 0  // passed tests
       var d = 0  // discarded tests
       var res: Result = null

Cheers,
Toralf