allow specifying seed so that `generate` can be repeatable. about swiftcheck

[CodaFi]

I'm hesitant to add a feature that allows users to control the seed beyond the initial seed in the checker arguments structure. It's an implementation detail of the underlying RNG that it exists at all.

[CodaFi]

I'll note that if you have a deterministic idea of how something should be constructed, write out the function that describes it.

[johanatan]

Here it is:

extension Gen {
    /// Initializes the generator's internal Random Number Generator with a specified seed.
    public func withSeed<S : BinaryInteger>(_ seed : S) -> Gen<A> {
           ...
    }
}

If you do not want to implement this, can you provide a workaround or some other way to achieve the deterministic/reproducible behavior that I'd like to see for this case?

i.e., the case is:
"where the length of an array that I need to generate will be determined dynamically by the code under test". i.e., SwiftCheck generates some values, I run some of my code-under-test and verify that it is correct. Then I need to generate an array of values the length of which is functionally determined by the already verified correct output of the code-under-test.

[CodaFi]

I’m still confused by this setup. You have a property that generates a size to pass to a generator so it can build you an array of that size to feed to your program, right? Where along this chain do you capture the RNG’s seed to make it generate “deterministic” outputs?

[johanatan]

As I mentioned previously, the seed would be passed as input to the property via an Arbitrary.Int. This way there would still be randomness (otherwise I could just fix the values via the repro CheckerArg).

[CodaFi]

But why use an arbitrary integer to generate a seed value at all? Could you show me a little pseudo-code of your setup, I think it'll help me comprehend this better.

[johanatan]

So that it will be random yet repeatable if needed.

[johanatan]

Here is some pseudo-code for it:

property("findCorrelations produces correct results.") <-
    forAll(CustomEntityArrayGen(upperBound: 100),
           CustomEntityArrayGen(upperBound: 75),
           Int.arbitrary) {

        let (array1, array2, checkIndicesSeed) = $0
        let res: Array<AnotherCustomEntity> = CodeUnderTest.f(array1, array2)

        // res is not easily predictable without generating a lot more data and doing
        // basically a reimplementation of `CodeUnderTest.f` in the tests.
        // res is also not able to be exhaustively checked.
        // thus, I'd like to generate a random sampling of indices into `res`
        // to perform swaps on and see if some function g(swapped_res) is less
        // than g(res). (f is supposed to produce a res that maximizes g(res)).

        let last = res.index(before: res.endIndex)
        let indexGen = Gen<Int>.choose((0, last))

        // note that i'm using `variant` in the following. I'd prefer to use `withSeed`
        // as that would make the code fully deterministic/repeatable.
        //

        let swapGen = indexGen.flatMap { (i) in indexGen.suchThat { $0 != i }.flatMap {
            Gen.pure((i, $0))
        }.variant(checkIndicesSeed)

        let numSwaps = h(res.count) // numSwaps is some function of res.count

        // generate numSwaps swaps with swapGen, do the swaps on res and
        // check g() values thereof.
}

[johanatan]

@CodaFi Have you had enough time to digest that code yet?

[CodaFi]

@johanatan I'm sorry for neglecting this. I'm a student in university and it's exam season. I owe you a comprehensive reply, and I simply haven't found the time to do so.

[johanatan]

Oh, gotcha. No worries. Just wanted to make sure it was still slated for attention at some point.

[CodaFi]

Let me try something quick and dirty: It seems like you have an unexpressed dependency between the collection generator and the indices generator:

property("findCorrelations produces correct results.") <-
    forAll(CustomEntityArrayGen(upperBound: 100),
           CustomEntityArrayGen(upperBound: 75),
           Int.arbitrary) {

        let (array1, array2, checkIndicesSeed) = $0
        let res: Array<AnotherCustomEntity> = CodeUnderTest.f(array1, array2)

        // res is not easily predictable without generating a lot more data and doing
        // basically a reimplementation of `CodeUnderTest.f` in the tests.
        // res is also not able to be exhaustively checked.
        // thus, I'd like to generate a random sampling of indices into `res`
        // to perform swaps on and see if some function g(swapped_res) is less
        // than g(res). (f is supposed to produce a res that maximizes g(res)).

        let indexGen = Gen.fromElements(of: res.indices)
        return forAll(Gen.zip(indexGen, indexGen)) { (i, j) in
          // Apply swap, check properties
        }
}

[johanatan]

Ah, so forAll can be nested? Yea, that should work if so.

[CodaFi]

Going to close this due to inactivity. Please feel free to reopen it if there is an update.