[RFC] Subtest directory orginization about autotest-docker

[cevich]

@Lorquas @ldoktor @jzupka what are your thoughts?

[jzupka]

It is no bad idea. In docker program has integration tests integration_cli tests
Why not?
I have only problem with subtest group bugzilla, it could help docker developers,
but removing integration_cli test from integration_cli folder, it could make mess, though.
If we have way how to start separate subsubtest it could solve bugzilla problem.
We could write to every bugzilla command which start specific subtest
autotest-local integration-cli/run/run_simple to keep it simple for developers.

[Lorquas]

I like the idea of narrowing the folders down into different test groups, and we should go for it, but I think the bugzilla group is too generic. When the bugzilla plugin starts working, there will be bug blockers against nearly every test eventually as the docker project matures. And once docker becomes feature complete for a release, most of the automation tasks against it tend to be bug related. At that point a bugzilla folder becomes the same kind of dumping ground as a utils kind of thing.

This could be great though for when tests start to get very in depth.

[cevich]

Ahh yeah, bugzilla won't work, I agree. I'm even afraid "integration" and "integration_cli" overlap way too much. There's going to be a problem also if we needed to split sub-subtests across multiple categories. Probably the only fair way to do this is how we have it now, more-or-less split by interaction-interface (CLI for now) then docker command (more or less). I'll close this then, but if anyone else has an idea to share, please re-open.