kyrus / python-junit-xml Goto Github PK

Per an example provided in the docstring, it seems the top level <testsuites> element should have aggregated attributes of all of its children <testsuite> elements.

As a result, instead of seeing a <testsuites> element that looks like:

<testsuites errors="1" failures="1" tests="4" time="45">

You would see a <testsuites> element that looks like:

<testsuites>

Is this a bug?

As a software engineer, I would like to use a class method to set a disabled test rather than setting the is_enabled class member to True.

Much like #43 but again in October 2019. Let me know if I can help in any way.

The <testsuite> element supports errors too:

<error></error> => optional information, in place of a test case - normally if the tests in the suite could not be found etc.

Maybe TestSuite could also have an add_error_info()?

"time" is available in testcase level, while it is missed in testsuite element, probably add line like

    test_suite_attributes['time'] = str(sum(c.elapsed_sec for c in self.test_cases if c.elapsed_sec))        

python-junit-xml/junit_xml/init.py has an example file with tests=3 but there are 4 testcase elements.

Is this a valid generated xml file? If so, what does it mean?

I want to judge a failure testcase using junit-xml v1.0, how can I make the failure title is not 0 in testsuite?

The module is not able to handle input with non-ASCII characters. If unicode is passed, the str() conversion on most of the parameters throws UnicodeEncodeErrors as the Python default encoding of ASCII is not able to handle the non-ASCII characters.

If UTF-8 encoded strings are passed, etree's element.tostring() method is expecting unicode and tries to encode it:

  File "C:\python27\lib\site-packages\junit_xml\__init__.py", line 169, in to_xml_string
    xml_string = ET.tostring(xml_element, encoding=encoding)
  File "C:\python27\lib\xml\etree\ElementTree.py", line 1126, in tostring
    ElementTree(element).write(file, encoding, method=method)
  File "C:\python27\lib\xml\etree\ElementTree.py", line 820, in write
    serialize(write, self._root, encoding, qnames, namespaces)
  File "C:\python27\lib\xml\etree\ElementTree.py", line 939, in _serialize_xml
    _serialize_xml(write, e, encoding, qnames, None)
  File "C:\python27\lib\xml\etree\ElementTree.py", line 932, in _serialize_xml
    v = _escape_attrib(v, encoding)
  File "C:\python27\lib\xml\etree\ElementTree.py", line 1090, in _escape_attrib
    return text.encode(encoding, "xmlcharrefreplace")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 4: ordinal not in range(128)

The workaround of changing Python's default encoding is considered evil and not always an option.
It could be possible to internally only use unicode and check and convert all input.

The readme and front page say

See the docs and unit tests for more examples.

However I cannot find any docs and the unit test file https://github.com/kyrus/python-junit-xml/blob/master/test_junit_xml.py doesn't have any comments in the code on what does what, some of it is self explanatory but not all IMO.

Hi there!

I'm trying to use this library, and I'd like to know how I tell it to mark a test as failed, errored, or skipped. The Readme says "See the docs and unit tests for more examples." but I can't find any doc and the cases mentioned above don't seem to be covered in tests from a quick glance.

Where can I go to find out more?

Thanks!

I run the sample code according to the README, and got the output like

$ ./xunit_report.py
<?xml version="1.0" ?>
<testsuites>
        <testsuite errors="0" failures="0" name="my test suite" tests="1">
                <testcase classname="some.class.name" name="Test1" time="123.345000">
                        <system-out>I am stdout!</system-out>
                        <system-err>I am stderr!</system-err>
                </testcase>
        </testsuite>
</testsuites>

Is it the new feature or I can disable the root element testsuites ?

Junit support error messages per testcase
cf http://stackoverflow.com/questions/4922867/junit-xml-format-specification-that-hudson-supports

add_*_info should add multiple messages instead of just adding one error/failure message

Hi @brianebeyer,
or whoever has access to Pypi for junit-xml. :) Would it be possible to make a new release? The previous one is almost one year and as far as I see there are a few new features (which I would like to use :) ).

Thanks, best
Karel

Jenkins parser appears to break when
test_case_attributes["time"] = "%f" % case.elapsed_sec
ends up with ludicrous number of decimal places.
Suggest using a format limit:
test_case_attributes["time"] = "%.3f" % case.elapsed_sec
(assuming milliseconds is considered enough precision)

The readme suggests use of the deprecated TestSuite.to_xml_string() method instead of the new to_xml_report_string() method from #62.

see https://github.com/kyrus/python-junit-xml/blob/master/README.rst#using

Hello there, good developed tool! Thanks for making this easy.

I'm using junit-xml on my jenkins automation. I'm trying to format output is add_xxx_info. '\n' doesn't work for me, and I may not read carefully enough, I don't see how you handle outputs like failure or error output other than writing in certain text content and append the object in testcase.

To visualize what I'm saying:
What I expected
<system-out>
This is the first line
This is the second line
</system-out>
What I got
<system-out>This is the first line\nThis is the second line\n</system-out>

Please teach me how you handle formats of output.

Thank you,
ZD

Thanks for the tool. Its pretty awesome.

Is there a way to create multiple testsuite with this package ?

Thanks

I have observed this issue with junit-xml==1.8 and Jenkins 2.46.3.
In case it matters I'm using xUnit plugin to publish Junit results.

Steps to reproduce

Create a test case with a multi-line output or traceback, e.g.

stdout = "line1\nline2"
traceback = "line1\nline2"
tc = TestCase(name=name, classname=classname, stdout=stdout)
tc.add_error_info(output=traceback)

Expected behavior

In the test case details displayed in Jenkins the multi-line values are shown like

line1
line2

Based on the formatting of test cases in the xunit plugin, I assume the proper tag to be generated should be:

			<system-out>
				line1
                                line2
			</system-out>

Actual behavior

The first line has an extra indentation:

            line1
line2

That is caused by the indentation given to text of the relevant tag:

			<system-out>
				line1
line2
			</system-out>

Could you please publish tags here so it would allow tracking on what version of junit-xml contains what fixes?

Also with release 1.9 it would be awesome if pypi contain the source tarball too not just a wheel.

The released version 1.3 from PyPI fails with TabError: inconsistent use of tabs and spaces in indentation. As the issue seems to be fixed in master, I would suggest to perform a new release. The issue can e.g. be seen in https://travis-ci.org/blue-yonder/devpi-builder/jobs/32638794.

This is a great project, I'm curious if there's support for handling parameterized tests much like you'd see running the following example...

#boolParamTest.py
import pytest

@pytest.fixture( params=[True, False, True, True, False] , ids=['BP1' ,'BP2', 'BP3', 'BP4' , 'BP5'] )
def boolParam( request ) :
    return request.param

def test_params(boolParam) :
    assert boolParam

def test_FalseParams( boolParam ) :
    assert not boolParam

And here's the console output from running this simple test...

$ pytest boolParamTest.py --junitxml=boolParamTestResult.xml
================================================================================================================ test session starts ================================================================================================================
platform darwin -- Python 2.7.14, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
rootdir: /Users/me, inifile:
collected 10 items

testParams.py .F..FF.FF.                                                                                                                                                                                                                      [100%]

--------------------------------------------------- generated xml file: /Users/me/out.xml ----------------------------------------------------
===================================================================================================================== FAILURES ======================================================================================================================
_________________________________________________________________________________________________________________ test_params[BP2] __________________________________________________________________________________________________________________

boolParam = False

    def test_params(boolParam) :
>       assert boolParam
E       assert False

testParams.py:9: AssertionError
_________________________________________________________________________________________________________________ test_params[BP5] __________________________________________________________________________________________________________________

boolParam = False

    def test_params(boolParam) :
>       assert boolParam
E       assert False

testParams.py:9: AssertionError
_______________________________________________________________________________________________________________ test_FalseParams[BP1] _______________________________________________________________________________________________________________

boolParam = True

    def test_FalseParams( boolParam ) :
>       assert not boolParam
E       assert not True

testParams.py:12: AssertionError
_______________________________________________________________________________________________________________ test_FalseParams[BP3] _______________________________________________________________________________________________________________

boolParam = True

    def test_FalseParams( boolParam ) :
>       assert not boolParam
E       assert not True

testParams.py:12: AssertionError
_______________________________________________________________________________________________________________ test_FalseParams[BP4] _______________________________________________________________________________________________________________

boolParam = True

    def test_FalseParams( boolParam ) :
>       assert not boolParam
E       assert not True

testParams.py:12: AssertionError
======================================================================================================== 5 failed, 5 passed in 0.08 seconds =========================================================================================================

And here's my corresponding generated xml...

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="5" name="pytest" skips="0" tests="10" time="0.084">
    <testcase classname="testParams" file="testParams.py" line="7" name="test_params[BP1]" time="0.00147676467896"></testcase>
    <testcase classname="testParams" file="testParams.py" line="7" name="test_params[BP2]" time="0.00170993804932">
        <failure message="assert False">boolParam = False

    def test_params(boolParam) :
&gt;       assert boolParam
E       assert False

testParams.py:9: AssertionError</failure>
    </testcase>
    <testcase classname="testParams" file="testParams.py" line="7" name="test_params[BP3]" time="0.000910997390747"></testcase>
    <testcase classname="testParams" file="testParams.py" line="7" name="test_params[BP4]" time="0.000877141952515"></testcase>
    <testcase classname="testParams" file="testParams.py" line="7" name="test_params[BP5]" time="0.00109601020813">
        <failure message="assert False">boolParam = False

    def test_params(boolParam) :
&gt;       assert boolParam
E       assert False

testParams.py:9: AssertionError</failure>
    </testcase>
    <testcase classname="testParams" file="testParams.py" line="10" name="test_FalseParams[BP1]" time="0.00115871429443">
        <failure message="assert not True">boolParam = True

    def test_FalseParams( boolParam ) :
&gt;       assert not boolParam
E       assert not True

testParams.py:12: AssertionError</failure>
    </testcase>
    <testcase classname="testParams" file="testParams.py" line="10" name="test_FalseParams[BP2]" time="0.000971794128418"></testcase>
    <testcase classname="testParams" file="testParams.py" line="10" name="test_FalseParams[BP3]" time="0.00102472305298">
        <failure message="assert not True">boolParam = True

    def test_FalseParams( boolParam ) :
&gt;       assert not boolParam
E       assert not True

testParams.py:12: AssertionError</failure>
    </testcase>
    <testcase classname="testParams" file="testParams.py" line="10" name="test_FalseParams[BP4]" time="0.0014009475708">
        <failure message="assert not True">boolParam = True

    def test_FalseParams( boolParam ) :
&gt;       assert not boolParam
E       assert not True

testParams.py:12: AssertionError</failure>
    </testcase>
    <testcase classname="testParams" file="testParams.py" line="10" name="test_FalseParams[BP5]" time="0.00100708007812"></testcase>
</testsuite>

However this may not be a use case the tool needs to manage, the client could just as well structure things via the name attribute of the test case.

to_xml_string() example:

1. Nothing is being passed to the to_xml_string() function
2. Calling the function on the object "ts" instead of the TestSuite class. Errors if ts is already a list object.

Here is my version that works:
test_cases = [TestCase('Test1', 'some.class.name', 123.345, 'I am stdout!', 'I am stderr!')]
ts = [TestSuite("my test suite", test_cases)]
print(TestSuite.to_xml_string(ts))

to_file() example:

1. Calling the function on the ts object. Should be on the class.
2. function is "to_file()", not "write_to_file()" as listed.
3. Might want to include f.close() just for those who may not know they need to close the file

generated output does not produce valid xml for any XSD schema from

https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model//src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/

so it can't be used by xUnit jenkins plugin

block has unknown attribute "skipped",
"documentation" on http://llg.cubic.org/docs/junit/ specifies attribute "disabled" instead

Hello,
According to the PyPI documentation junit-xml supports Python 3.
Does that also means it supports python 3.11x?
if not, is there any plan of supporting it?

The tests are executed by pytest. Would it be ok if I converted the tests to pytest functional form instead unitest class form?

XML files created by the latest version of this package are not handled by xUnit versions newer than 2.0. In the xUnit change log it is specified that, for JUnit, the two schemas supported are Ant junit and Maven Surefire and the current output of this package does not conform to neither.

In principle, the output of junit-xml used to conform to the Maven schema, and this is the case for XML files created by 1.7 and earlier. But, in PR #42, a "disabled" attribute was added to test suite, apparently following the jUnit 10 XSD. However, it appears that there is no "disabled" attribute in that very same XSD file now. Perhaps the attribute was there at the time of the PR, nevertheless now this attribute breaks validation from xUnit.

Reverting to 1.7 can be used as a workaround for this issue at the moment but, long term, it would be great if this "disabled" attribute could be stripped from the generated XML files, thus restoring compatibility with xUnit.

Thanks for your assistance.

If "prettyprint=False", to_xml_string() converts the xml string to byte instead of string.

File "C:\Python33\Lib\site-packages\junit_xml-1.0-py3.3.egg\junit_xml__init__.py", line 149, in to_file
file_descriptor.write(TestSuite.to_xml_string(test_suites, prettyprint))
TypeError: must be str, not bytes

This is because of the following line in to_xml_string():
xml_string = ET.tostring(xml_element)

which causes ET.tostring() to convert to BytesIO instead of StringIO. The fix is:
xml_string = ET.tostring(xml_element, encoding="unicode")

When issuing the command: pip install junit-xml, it actually downloads the macosx-10.8-intel.tar.gz package which containts no setup.py, hence failing the installation.

Note that if you point pip to the good url like so:
pip install https://pypi.python.org/packages/source/j/junit-xml/junit-xml-1.1.tar.gz#md5=86158819d747db99d1d2d6a73ab90845
the package is correctly installed.

This "issue" needs to be fixed as it is kinda annoying ;)

👋 I wonder if this project is still maintained, especially regarding this outstanding issue. Thanks!

I would like to change the interface of this lib. I can present a first step in a pr.

I find it weird that to_xml_string and to_file are static methods for TestSuite. Their input is a list of TestSuite objects so they don't act on a TestSuite object. I would like to move them out of the TestSuite class to separate functions that act on a list of TestSuite objects. An alternative would be to introduce a new class called TestSuites that can be instantiated with a list of TestSuite objects.

I actually would like to remove the to_file method entirely. It just writes the xml string to an open file. I don't see why that would be the responsibility of this lib.

I think build_xml_doc should not be exposed outside of the lib. TestCase and TestSuite should just be simple holders of data and we can turn a list of TestSuite objects to a xml string. But I can present a suggestion for that in a later pr. I have only found one repo on github that uses build_xml_doc (https://github.com/CyberGRX/questions-three).

Unicode support in Python 3 is very different from how it works in Python 2. If the module should truly work across Python 2 and 3, it needs to be modified to operate differently under each version.

Just noticed, that some files are missing in the junit-xml-1.4.tar.gz archive from PyPi:

• test suite
• tox.ini
• …

test_suite_attributes['time'] = str(sum(c.elapsed_sec for c in self.test_cases if c.elapsed_sec))

If test case run parallel, just sum elapsed_sec of all the cases is not make sense

In testsuite init function, may you add time?

Hi
I develop python within visual studio and I make use of the 64bit python interpreter because it crashes less in visual studio. How can I make junit-xml available in 64bit python environment.

Thanks

Mudasir

It would be nice to have the (PyPI) releases also on Github. Here's a tutorial which suggests to use 'git tag':

http://peterdowns.com/posts/first-time-with-pypi.html

This would also help as a mental trigger to bump the version.

Just saw this, and wondered:
f2daf35#commitcomment-24140156

The 1.9 release on PyPi is not available as an sdist, only as a wheel. Providing both would be wonderful, to make it easier to distributions to update to the new versions.