Problem Description: The execution order of Test Suites in a project is not proper. For example when there are two test suites and another script which is not mapped to any Test Suite in a project, the execution order is as follows: First Test Suite 2 is executed, then standalone script and then the test suite 1 is executed.
Steps:
- Create a project Example
- Add a script init.py to the project
- Add a test suite1 to the project after the init.py
- Add 3 test scripts under Test Suite 1
- Add a test suite 2 to the project after Test Suite 1
- Add 3 test scripts under Test Suite 2
- Save the project and run it
Expected Result:
The order of execution should be:
- init-py
- Test Suite1
- Test Suite2
Actual Result:
The order of execution actually is:
- Test Suite 2
- init.py
- Test Suite 1
Attached is the screenshot:
![execution](https://camo.githubusercontent.com/036fbf605d09d6ffab650884f18b8a230c950796315dd81b83e16ec837291cd0/68747470733a2f2f662e636c6f75642e6769746875622e636f6d2f6173736574732f343239313536372f3437373036312f61323462383366612d623764342d313165322d396563382d3537616338393339613263642e6a706567)
Example.xml:
false
false
PROJECT
false
tb1
Run_Number
1
/home/user/twister/demo/testsuite-python/init.py
Runnable
true
Running
true
TESTSUITE1
tb1
/home/user/twister/demo/testsuite-python/test_pexpect_ftp.py
Runnable
true
Running
true
/home/user/twister/demo/testsuite-python/test_pexpect_ssh.py
Runnable
true
Running
true
/home/user/twister/demo/testsuite-python/test_py_globals1.py
Runnable
true
Running
true
TESTSUITE2
tb1
/home/user/twister/demo/testsuite-python/test_py_globals2.py
Runnable
true
Running
true
/home/user/twister/demo/testsuite-python/test_py_globals3.py
Runnable
true
Running
true
/home/user/twister/demo/testsuite-python/test_py_printnlogs.py
Runnable
true
Running
true
## Log File:
Py debug: For EP EP-1001, CE Server returned a new status: running.
EP debug: Received start signal from CE!
TC debug: TestCaseRunner started with User: user ; EP: EP-1001.
TC debug: Connected to proxy, running tests!
Downloading library TestBed.py
...
Downloading library Threads.py
...
Downloading library TscFtp.py
...
Downloading library TscTelnet.py
...
===== ===== ===== ===== =====
Starting suite 102:TESTSUITE2
===== ===== ===== ===== =====
<<< START filename: 1004:/home/user/twister/demo/testsuite-python/test_py_globals2.py
>>>
Debug: dependancy =
, prereq =False
, optional =False
...
Setting 2 functions, 2 classes and 2 instances, for using in the next tests!
File /home/user/twister/demo/testsuite-python/test_py_globals2.py
returned PASS
. <<<
<<< END filename: 1004:/home/user/twister/demo/testsuite-python/test_py_globals2.py
>>>
<<< START filename: 1005:/home/user/twister/demo/testsuite-python/test_py_globals3.py
>>>
Debug: dependancy =
, prereq =False
, optional =False
...
*** All functions and classes are pointers, defined in the previous test!!!
Calling function 1
Some function #1
Calling function 1
Some function #2
Calling class 1
<builtin.Class1 instance at 0x8e1274c>
Calling class 2
<Class2 object at 0x8e1274c>
Checking instance 1
<builtin.Class1 instance at 0x8e1268c>
Checking instance 2
<builtin.Class1 instance at 0x8e1268c>
File /home/user/twister/demo/testsuite-python/test_py_globals3.py
returned PASS
. <<<
<<< END filename: 1005:/home/user/twister/demo/testsuite-python/test_py_globals3.py
>>>
<<< START filename: 1006:/home/user/twister/demo/testsuite-python/test_py_printnlogs.py
>>>
Debug: dependancy =
, prereq =False
, optional =False
...
TestCase:test_py_printnlogs.py starting
TEST_PY_PRINTNLOGS.PY: working 0...
TEST_PY_PRINTNLOGS.PY: working 1...
TEST_PY_PRINTNLOGS.PY: working 2...
TEST_PY_PRINTNLOGS.PY: working 3...
TEST_PY_PRINTNLOGS.PY: working 4...
TEST_PY_PRINTNLOGS.PY: working 5...
TEST_PY_PRINTNLOGS.PY: working 6...
TEST_PY_PRINTNLOGS.PY: working 7...
TEST_PY_PRINTNLOGS.PY: working 8...
TEST_PY_PRINTNLOGS.PY: working 9...
File /home/user/twister/demo/testsuite-python/test_py_printnlogs.py
returned PASS
. <<<
<<< END filename: 1006:/home/user/twister/demo/testsuite-python/test_py_printnlogs.py
>>>
===== ===== ===== ===== =====
Starting suite 100:PROJECT
===== ===== ===== ===== =====
<<< START filename: 1000:/home/user/twister/demo/testsuite-python/init.py
>>>
Debug: dependancy =
, prereq =False
, optional =False
...
I am init file.
I am not doing anything special, just printing some variables.
Hello, user user !
Test Bed: tb1
Exec process: EP-1001
Suite: PROJECT
File: /home/user/twister/demo/testsuite-python/init.py
File /home/user/twister/demo/testsuite-python/init.py
returned PASS
. <<<
<<< END filename: 1000:/home/user/twister/demo/testsuite-python/init.py
>>>
===== ===== ===== ===== =====
Starting suite 101:TESTSUITE1
===== ===== ===== ===== =====
<<< START filename: 1001:/home/user/twister/demo/testsuite-python/test_pexpect_ftp.py
>>>
Debug: dependancy =
, prereq =False
, optional =False
...
Connected to openbsd.sunsite.ualberta.ca.
220 openbsd.srv.ualberta.ca FTP server ready.
anonymous
331 Guest login ok, send your email address as password.
230- Welcome to ftp.openbsd.org at the University of Alberta
230- in Edmonton, Alberta, Canada.
230- For other mirror sites visit http://www.openbsd.org/ftp.html
230-
230- _____ ____ _____ _____
230- / ___ \ | _ \ / | __
230- / / / / ___ ____ | |) | (___ | | | |
230- / / / / __ / _ / __ | _ < ___ | | | |
230- / // / // / __/ / / /| |) |__) | || |
230- ___/ .**/**// // |___/|/|/
230- //
230- | . The proactively secure Unix-like
230- . |L /| . Operating System.
230- _ . |\ | --+./| . Please visit the OpenBSD web site
230- / ||| Y J ) / |/| ./ at http://www.openbsd.org/
230- J |)'( | F
.'/
230- -<| F __ .-< All transfers are logged, if you don't
230- | / .-'. . /-. L___ like this policy, disconnect now! 230- J \ < \ | | O\|.-' 230- _J \ .- \/ O | | \ |F 230- '-F -<_. \ .-'
-' L__ OpenBSD is available for order!
230- __J _ . >-' ).. |-' You can order OpenBSD CD's from
230- -|.' /_. \_| F http://www.openbsd.org/orders.html. 230- /.- . _.< CD sales are very important to support 230- /' /.' .'
\ the continued development of the project.
230- /L /' |/ _.-'-
230- /'J ___.---'|
230- |\ .--' V | .
230- |/.
-. ._) 230- / .-.\ 230- VK \ (
230- `.
230-
230- DO NOT mirror openbsd from this site! use one of the
230- "second level mirrors" listed at http://www.openbsd.org/ftp.html
230- instead of this site. If you mirror from this site you will lose
230- access to it.
230-
230- E-mail comments, questions, trouble reports, and complaints
230- to [email protected]. Please drive safely.
230-
230 Guest login ok, access restrictions apply.
Remote system type is UNIX.
Using binary mode to transfer files.
cd /pub/OpenBSD/
250 CWD command successful.
ls
500 Illegal PORT rejected (address wrong).
ftp: bind: Address already in use
File /home/user/twister/demo/testsuite-python/test_pexpect_ftp.py
returned PASS
. <<<
<<< END filename: 1001:/home/user/twister/demo/testsuite-python/test_pexpect_ftp.py
>>>
<<< START filename: 1002:/home/user/twister/demo/testsuite-python/test_pexpect_ssh.py
>>>
Debug: dependancy =
, prereq =False
, optional =False
...
=== Connecting to SSH ===
Exception:
Traceback (most recent call last):
File "/home/user/twister/client/executionprocess/TestCaseRunner.py", line 553, in
result = current_runner._eval(str_to_execute, globs, args)
File "/home/user/twister/client/executionprocess/TestCaseRunnerClasses.py", line 252, in _eval
execfile(fpath, globs_copy)
File "/home/user/twister/.twister_cache/EP-1001/test_pexpect_ssh.py", line 57, in
_RESULT = test()
File "/home/user/twister/.twister_cache/EP-1001/test_pexpect_ssh.py", line 30, in test
child.expect('user@localhost:', timeout=5)
File "/usr/lib/python2.7/dist-packages/pexpect.py", line 1311, in expect
return self.expect_list(compiled_pattern_list, timeout, searchwindowsize)
File "/usr/lib/python2.7/dist-packages/pexpect.py", line 1325, in expect_list
return self.expect_loop(searcher_re(pattern_list), timeout, searchwindowsize)
File "/usr/lib/python2.7/dist-packages/pexpect.py", line 1409, in expect_loop
raise TIMEOUT (str(e) + '\n' + str(self))
TIMEOUT: Timeout exceeded in read_nonblocking().
<pexpect.spawn object at 0x8e7e68c>
version: 2.3 ($Revision: 399 $)
command: /usr/bin/ssh
args: ['/usr/bin/ssh', 'user@localhost']
searcher: searcher_re:
0: re.compile("user@localhost:")
buffer (last 100 chars): 2013 from localhost
ailas: command not found
�]0;user@user-VirtualBox: �user@user-VirtualBox:$
before (last 100 chars): 2013 from localhost
ailas: command not found
�]0;user@user-VirtualBox: �user@user-VirtualBox:$
after: <class 'pexpect.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 5787
child_fd: 5
closed: False
timeout: 30
delimiter: <class 'pexpect.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
File /home/user/twister/demo/testsuite-python/test_pexpect_ssh.py
execution FAILED
. <<<
<<< END filename: 1002:/home/user/twister/demo/testsuite-python/test_pexpect_ssh.py
>>>
<<< START filename: 1003:/home/user/twister/demo/testsuite-python/test_py_globals1.py
>>>
Debug: dependancy =
, prereq =False
, optional =False
...
Root level params: ['NE1']
All params for Level_1: False
All params for Level_1/Level_2: False
First level global #1: False
Will set new value to:: value_13_804a31eef05f1f6267396cb0e9
Check value changed:: False
Will set new value to:: value_16_2b592e538cdc0b55ab0ddc038b75a14a
Check value changed:: False
Will set new value to:: value_8_d261c39f77b2ba1a
Check value changed:: False
Will set new value to:: value_15_f942f395f7a5099253bf6512de869b
Check value changed:: False
Will set new value to:: value_11_7e375687ec981c86f8529d
Check value changed:: False
Will set new value to:: value_7_82ecdb4118be2a
Check value changed:: False
Will set new value to:: value_14_e8f78dd9587511e4639b3d80c88d
Check value changed:: False
Will set new value to:: value_16_7646dd0db526c0f621ba6047af64935e
Check value changed:: False
Will set new value to:: value_18_8d7b807e1c1c1bdce14f77376809974f85fb
Check value changed:: False
Will set new value to:: value_6_c66dcdff36c1
Check value changed:: False
File /home/user/twister/demo/testsuite-python/test_py_globals1.py
returned PASS
. <<<
<<< END filename: 1003:/home/user/twister/demo/testsuite-python/test_py_globals1.py
>>>
===== ===== ===== =====
. . . All tests done . . .
===== ===== ===== =====
EP debug: Successful run!
Py debug: For EP EP-1001, CE Server returned a new status: stopped.