Terraform Compliance v1.3.31, Alternative text bug? about cucumber-expressions

[jenisys]

@j4t1nd3r
Cucumber is generic. The stuff what you are asking for is from a technical or automation domain.
That would be your step implementation in the Terraform technical domain that provides this implementation.
Because somehow you need to retrieve the vm_size. This kind of things are normally provided in step-libraries that are using other technical domain / automation domain library(s) to do the stuff.
Otherwise, the comparison if a value is one of a list of several is normally provided by assert-matchers, like Hamcrest.

[j4t1nd3r]

As a workaround. I got a suggestion from a colleague which works:

Feature: vm compliance

    Scenario: vm Sku compliance
        Given I have azurerm_virtual_machine defined
        Then it must have "vm_size"
        And its value must match the "^(Standard_DS1_v1|Standard_DS1_v2)$" regex

[aurelien-reeves]

I guess there is a misunderstanding of alternative text here.

TL;DR: alternatives appear in your step definitions in javascript/java/ruby/whatever. Not in your steps in your feature files.

The alternatives appear in the step definition (in your code), but the step itself (in your feature files) must pick one of the alternatives.

In your case, in your code, you would declare your step definition like that (in javascript for example, but whatever your language, your expression may look like the following):

then('its value must be Standard_DS1_v2/Standard_DS2_v1', async function () {
})

And in your feature file, you would use it like the following:

Feature: vm compliance

    Scenario: vm Sku compliance
        Given I have azurerm_virtual_machine defined
        Then it must have "vm_size"
        And its value must be Standard_DS1_v2

Or

Feature: vm compliance

    Scenario: vm Sku compliance
        Given I have azurerm_virtual_machine defined
        Then it must have "vm_size"
        And its value must be Standard_DS2_v1

But not both at the same time. This is the idea of alternative texts: you declare a single step definition but you can use it with several alternatives

[aurelien-reeves]

So, I close the issue
But if you think this is a misunderstanding from myself, please, reopen it!

[j4t1nd3r]

HI @aurelien-reeves, I understand what you're saying.

Is there a way to to use cucumber scenario to check a value against a white list? Such as here I would like to see if vm_size against a white list of allowed sizes?

[j4t1nd3r]

I figured it!

Feature: vm compliance

    related resources: azurerm_virtual_machine

    Background: vm resource specific 
        Given I have azurerm_virtual_machine defined

    Scenario Outline: vm Sku compliance
        Then it must have "vm_size"
        And its value must match the "^Standard_DS*" regex
        And its value must not be <sku>

        Examples:
        | sku               |
        | Standard_D4_v2    |
        | Standard_D5_v2    |

[aurelien-reeves]

Great that you figured it out!

However, caution. Here's your first version (I've removed the "outline" keyword as it is not needed here):

Feature: vm compliance

    related resources: azurerm_virtual_machine

    Background: vm resource specific 
        Given I have azurerm_virtual_machine defined

    Scenario: vm Sku compliance
        When it has "vm_size"
        Then it must have "vm_size"
        Then its value will contain one of the following sizes
        | Standard_DS1_v2   |
        | Standard_DS2_v2   |

And the new one:

Feature: vm compliance

    related resources: azurerm_virtual_machine

    Background: vm resource specific 
        Given I have azurerm_virtual_machine defined

    Scenario Outline: vm Sku compliance
        Then it must have "vm_size"
        And its value must match the "^Standard_DS*" regex
        And its value must not be <sku>

        Examples:
        | sku               |
        | Standard_D4_v2    |
        | Standard_D5_v2    |

There is a really big difference with those two versions. The expectation are absolutely not the same.

The first version, your scenario is generating a single test with a step "its value will contain one of the following sizes" wich has a single parameter wich contains your two values Standard_DS1_v2 and Standard_DS2_v2. It is up to your code in the step definition to make sure the tested value match the expectation.

The second version, your scenario outline is generating two distinct tests. The first one will then use the step "its value must not be " with sku being Standard_D4_v2, the second one sku will be Standard_D5_v2. This is definitely something different than the first version, and what you seem to actually expect.

I encourage you to reach the cucumber documentation to make sure you are fully comfortable with it. It looks you are specifically focused on feature files, so, here's a link to the gherkin reference itself: https://cucumber.io/docs/gherkin/reference/

[j4t1nd3r]

I did notice I am missing a character in my values for the must not match which I have fixed below.

You are right, I am not writing any step definitions and working exclusively in feature files. So I will go with the second example you have posted. Thanks. I will also check out the link.

Feature: vm compliance

    related resources: azurerm_virtual_machine

    Background: vm resource specific 
        Given I have azurerm_virtual_machine defined

    Scenario Outline: vm Sku compliance
        Then it must have "vm_size"
        And its value must match the "^Standard_DS*" regex
        And its value must not be <sku>

        Examples:
        | sku                |
        | Standard_DS4_v2    |
        | Standard_DS5_v2    |

[aurelien-reeves]

I would actually advice something like this:

Feature: vm compliance

    related resources: azurerm_virtual_machine

    Background: vm resource specific 
        Given I have azurerm_virtual_machine defined

    Scenario: vm Sku compliance
        Then it must have "vm_size"
        And its value must match the "^Standard_DS*" regex
        And its value must not be "Standard_DS4_v2"
        And its value must not be "Standard_DS5_v2"
    

There is actually a big difference here as your feature actually has two distinct tests while my suggestion has only one.

[j4t1nd3r]

Thnx. I will be adapting both methods depending on the use case!