Should we store the "actual" request data instead of/as well as the "expected" in the pact file? about pact-specification

[bethesque]

I think this raises the question "what sort of testing do we want pact to support?".

IMO (so far, I may be persuaded otherwise!), the best use of pact is to test the provider client in an isolated test. If every call to your provider goes through your client class, and you have tested every method of your client class then

1. you should be able to have confidence that your end to end integration will work
2. you avoid the combinatorial explosion problem that happens when you introduce integration tests - you have tested each scenario once and only once, meaning your tests scale linearly.

If your client returns a model, then all the other tests that interact with the client only need to know about the model, and that can be verified against the model that the client returns in the pact test by using the same factory to produce both the model in the pact spec, and the model that is returned in the test where the client is stubbed.

So, if I don't like the idea of using pact in end-to-end Capybara style tests, then how do I think we should run Capybara style tests? I haven't worked that out yet...

Maybe there should be two pact modes? One, the current way, where we require that the entire request is specified, and we record the expected request in the pact, and Two, where you specify the minimum request data required to uniquely identify the request, and the response, and then record the entire request in the pact.

[thetrav]

I don't think we need to consider testing the whole consumer as an integrated codebase to answer this question.

Rather we should answer the question of whether we want to support the consumer specifying non-strict matchers in the requests they send. (which somewhat contradicts Postel's law)

If the answer to that is yes (I can see it being useful for library generated headers, security tokens url's etc) then Ron's solution is the best I've seen for having a value to use in the provider test.

I'm trying to think of where non-strict consumer data would be of significant value and not coming up with many answers.

I can't see how a test with the eventId known in advance would be a problem.
I don't think we really want to support random test data, that's more appropriate for a Janus(quickcheck) type library.

[thetrav]

Timestamps and third party security libraries were the first thing I thought of, however my general feeling is that time should be stubbed in all tests and third party security libraries are still generally deterministic

[bethesque]

Why do you think that we shouldn't consider testing the whole consumer? Testing a rich client UI that uses a JSON REST API using capybara is a pretty typical use case.