Separate service testing from package testing and only report package testing result on README. about emodnetwfs

[annakrystalli]

Of course, if the services actual tests are based on fail we will still have failures in the package. We could look into mocking responses to test post request processes.

[salvafern]

Hi @annakrystalli,

Here some ideas but I'm very open to discussion 😊

Test service in a different GitHub Action

I agree but ideally once #24 is solved (e.g. read the available endpoints from the Central Portal CSW or similar) we wouldn't need to test the services at all. Do you mean that we should test the services in a different GitHub Action as a temporal fix or also for the long term?

Mocking responses for testing sounds like a good idea for me. This way we what the package is supposed to do without relying on the services. But these could be part of the regular testing of the package and no need for a different GitHub action I think.

Report package result on README

I'm not completely sure what do you mean by this. Something that we have been struggling is that the build can fail because there are examples in the README and the vignette that rely on an specific service. (e.g. the build yesterday). How would you show examples without risking that they make the build fail?

Some random ideas:

1. Show examples but do not actually run (eval = FALSE)
2. Somehow make the examples depend on the services available at the Central Portal CWS
3. The examples use mock responses.

I kinda like more option 2 the most so far.

[annakrystalli]

Test service in different GA

Actually you are right. Once we sort out #24, this will be unnecessary so it's probably not worth the effort.

Report package results in README

sorry I wasn't very clear! I meant the R CMD CHECK status badge on the README should not reflect the status of the servers in any way but rather the code. Indeed the examples in vignettes and the README you mention that are server status dependant but can cause our checks to fail are also a similar problem.

Solution

In all honesty I haven't made best use of tools available or best practice of testing API wrappers (sorry still learning!) and agree that mocking is that right way forward. Indeed looking at package httptest https://enpiar.com/r/httptest/, I think it has all the functionality we need! And as we can capture responses when the server is working and cache it, we can try and use such cached objects for our vignettes and README too! 😃

So moving forward I think we should:

• Overhaul testing of our functions using mocking and httptest (this issue).
• Move testing of services to runtime rather than as part of testing suite (as per #24).

I'm happy to handle this issue. So do you want to look into #24 @salvafern or do you want me to handle both?

Also, happy to schedule a quick call to get on the same page and brainstorm some ideas if you want.

[salvafern]

Had a chat with @annakrystalli today:

• @annakrystalli: Ask to the rOpenSci community what do they do in these cases in other packages
• @annakrystalli: Use httptest to catch data in the README and vignettes.
• @annakrystalli: Add a job to check and update the cached data if possible. This will run approx. every month and it will be separated from the R-CMD-Check.

@salvafern will work on #24

[maelle]

What remains to be done here? httptest is now used.