Acceptance Testing about terraform-provider-ns1

[Zach-Johnson]

Yes, the acceptance tests are currently failing, I believe due to some API changes that were not reflected in this provider.
The exception may be Pulsar jobs - that may be due to lack of feature enablement on the account you are using to run the acceptance tests.
I'm working on getting fixes for this prioritized internally, thanks for raising this.

[zahiar]

I have the same issue, for me the following resources having failing acceptance tests:

• DNS SEC
• API Key
• Application
• Pulsar Job
• Record
• Team
• User
• Zone
• NotifyList - though I have a fix for this one, pull request has been raised: #214

[zahiar]

@Zach-Johnson Would it be possible to also have these acceptance tests run automatically when a pull request is opened? Similar to how the current test-pr workflow that runs unit tests, that could be expanded to run acceptance tests as well.

[Zach-Johnson]

@zahiar yeah this is definitely doable, we just don't have a lot of bandwidth for improvements on this provider right now.

[eravin-ns1]

We fixed most of the acceptance tests in v1.13.0, which was released today. The remaining test failures are actual bugs like issue 228 and we hope to fix them soon.

[eravin-ns1]

Note that some of the tests require having an account enabled for Pulsar, specific dedicated networks, or other paid features. I'm thinking we could adjust the names of the tests to flag the ones not likely to succeed for the "average" end user.

[zahiar]

@eravin-ns1 Thanks 🙏

One option could be to hide tests that rely on paid features behind a flag. So by default all free tests are run, and if you pass in a flag, then all free & paid tests are run. Just a thought to make it easier, given that most users won't have those paid features on their accounts.