When I run distaf from the main source directory, I get the following Python stack:

# ./main.py -c example.conf.yml 
Traceback (most recent call last):
  File "./main.py", line 117, in <module>
    main()
  File "./main.py", line 100, in main
    collect_tests()
  File "./main.py", line 36, in collect_tests
    __import__(iname.replace("/", "."))
  File "/opt/distaf/tests_d/geo_rep/test_geo_basic.py", line 4, in <module>
    @testcase("changelog-test-create")
  File "/opt/distaf/distaf/util.py", line 49, in decorator
    tc_config = get_testcase_config(func.__doc__)
  File "/opt/distaf/distaf/config_parser.py", line 33, in get_testcase_config
    config_dict = yaml.load(config_string)
  File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 69, in load
    loader = Loader(stream)
  File "/usr/lib64/python2.7/site-packages/yaml/loader.py", line 34, in __init__
    Reader.__init__(self, stream)
  File "/usr/lib64/python2.7/site-packages/yaml/reader.py", line 85, in __init__
    self.determine_encoding()
  File "/usr/lib64/python2.7/site-packages/yaml/reader.py", line 124, in determine_encoding
    self.update_raw()
  File "/usr/lib64/python2.7/site-packages/yaml/reader.py", line 178, in update_raw
    data = self.stream.read(size)
AttributeError: 'NoneType' object has no attribute 'read'

Removing the tests_d/geo_rep directory makes it possible to run the tests.

Right now whenever there is a failure in each test case, the result says FAIL and throws exception. That exception looks bit ugly and should be suppressed.

When there is a single node in the cluster, the peer validation fails because peer status fails.

I installed and ran distaf against two gluster server and its result is passed it seems but I could not see any gluster related info at the test nodes (gluster server).

I did ssh-copy-id for user fractalio which is not root user.

Do I need to do ssh-copy-id for root user ?

How to verify that the testcase has been really executed on the test nodes ?

Let me know, if you need any further info.

config.sh file content:

export MGMT_NODE="mgmt_node"

export NODES="192.168.1.244"

export CLIENTS="client0 client1"

export PEERS="192.168.1.245"

export GM_NODES="gm_master0 gm_master1"

export GS_NODES="gs_slave0 gs_slave1"

development1@development1-ME051 ~/Documents/distaf $ sudo python main.py

test_gluster_basic_test (main.gluster_tests) ... ok

Ran 1 test in 0.001s

OK

development1@development1-ME051 ~/Documents/distaf $ tail -f /var/log/tests/distaf_tests.log

2015-02-20 13:49:49,254 INFO gluster_basic_test Testing gluster volume create and mounting
^C

Packaging the management node in a docker container will help users quickly started with running the tests and contribution.

There should be guide to write and execute new test case. Please add proper documentation in the source code itself.

I have two questions need to inquiry:
1, when I perform the test case, prompt yaml is not found, yaml is what? How is generated?
2, util.py document, testcase, test_list, test_seq, test_mount, globl_configs, global_mode, tc value fill out there is no limit? Can you see you how to fill in?

Distaf is a distributed testing framework, but it can be done? What are the advantages? Whether can describe for me? I saw a test tool blockade yesterday, I know it can be done? But I see the distaf, I don't know what can it do?

Right now whenever there is a exception in any test case it is caught and only name of the exception is thrown. But this is not very friendly for debugging. So either don't catch the exception or find a way to print the entire stack trace in a better way.

Please add a license and copyright notice in all of the source files.

One of the QA guys reported a bug against Gluster/NFS. The problem was reproducible with quite a time-consuming test:

mount -t nfs storage.example.com:/volume /mnt
cd /mnt
if [ ! -e linux-3.19.tar.gz ]
then
    curl https://cdn.kernel.org/pub/linux/kernel/v3.x/linux-3.19.tar.gz > linux-3.19.tar.gz
fi

# abort on error
set -e

for i in {1..1000}
do
    tar -xvf linux-3.19.tar.gz
    sleep 180
    rm -rf linux-3.19
done

I would like to write a test-case for this, but it should not execute by default. It would be nice to have an option to mark a test-case with "tags: long-running" or something in the doc string so that tests like this can easily be skipped.

Because tests and code are split over multiple repositories, it would be good to have an option in the test metadata that can be used to only run the test if the version of the components is high enough. (Or run the tests, but do not include the result in the final PASS/FAIL result?)

Practical example:
I would like to add a test to distaf that tests for different mount options. The change in Gluster might not be merged or backported to all branches yet. It would be bad for distaf to fail for tests that are not expected to pass for certain versions.

get_config_data() in lib/util.py could be more elegant. Instead of a series of if/else statements, iterating a list of possible keys (along side the regular assignments) would be better IMO.