New E2E Test Framework Discussion about gloo

[sheidkamp]

Like:

• The reusable YAML definitions. Also makes it easy to set up a demo and then just use the resources for the tests.
• The "debug test" button in vscode

Dislike:

• Lack of a stack trace on failures
• Inability to focus tests

Suggestions for future improvements:

• There are patterns developing that could/should get standardized into reusable code
  • The association of manifests and resources to tests
  • Looping over the manifests and expected resources at Test/Suite setup and teardown
  • Inclusion of standard/default resources( like a curl/nginx pod or a petstore deployment) handled by at a high level
    • Will need to be moved to a common place first. I did this in gloo with the curl pod as a first step.
  • An interface that includes what seem to be the common components of the test suites (ctx, testinstallation, testify suite) that can enable some of the above
• Look at the types of yaml resources we are creating and (if appropriate) come up with a standard naming convention and directory structure inside /testdata
• Documentation around things like "what is a feature?"