Comments (8)
The problem is actually not in org.testng.reporters.XMLReporter but in SuiteRunner.runTest():
private void runTest(TestRunner tr) {
tr.run();
ISuiteResult sr = new SuiteResult(m_suite, tr);
m_suiteResults.put(tr.getName(), sr);
}
The result is of TestRunner execution is added to m_suiteResults using test name as key, which is incorrect because TestNG doesn't enforce against having multiple tests with same name within a single suite.
Solution:
- When reading the XML suite, only create a single XmlTest for multiple tests with same name? This might not work because user might specify different set of parameters with different tests.
- Start enforcing that tests within same suite will have unique names. This has potential of breaking existing tests for many people.
- Append random character or something like "_1" at end of test name if m_suiteResults.get(testName) returns non-null value. Not sure how this will go down with folks when it comes down to comparing input xmls and outputs from reporters and listeners
- Implement something where TestRunners can be merged ???
What do you suggest Cedric?
from testng.
I also thought about all these options and neither is really straightforward. The best place to do this is as far up stream as possible, i.e. 1), but the XML parsing needs to be updated in several places (because of all the m_current*** fields I am maintaining because of SAX).
from testng.
unless you see a real need for having multiple test tags with same name, we should maybe break the backward compatibility in this case by going with option 2. It's the cleanest and simplest solution. I don't see a lot of people running into this issue, otherwise it would have been reported earlier.
Otherwise, If you do go with option 1, won't you run into the issue with different set of parameters defined for different tests. How do you think that would be handled?
from testng.
Well, it's already undefined behavior so we wouldn't make things much worse, but yes, maybe we should just catch this in ContentTestNGHandler and abort if we detect two tags with the same name.
from testng.
ok. I'll make the change.
from testng.
Actually, a correction: we should catch this slightly higher than at XML parsing so that the fix will catch this both in XML and YAML documents.
from testng.
Should be fixed with nullin@7f570c9.
Opening a pull request now.
from testng.
I think this issue should be closed too.
from testng.
Related Issues (20)
- How to enable generateTestResultAttributes programatically using Java ? HOT 1
- Should not we use the lock everywhere suiteResult is used? HOT 1
- Associate a unique id with every test class object instantiated by TestNG
- Discrepancy with combination of (Shared Thread pool and Method Interceptor) HOT 1
- IInvokedMethodListener Iinvoked method does not return correct instance during @BeforeMethod, @AfterMethod and @AfterClass
- Please document project's PGP artifact signing keys HOT 2
- Why is my test case not retried but ignored after execution failure? HOT 5
- Possible to merge PR 3067 in 7.9.0 HOT 2
- Super class annotated with ITestNGListenerFactory makes derived test class throw TestNGException on execution HOT 1
- Test cases are being skipped when running in parallel a factory class containing dependent methods HOT 11
- Update from testng 7.9.0 to 7.10.0 break maven build with junit5 HOT 11
- Replacement API for `IClass.getInstanceHashCodes()` and `IClass.getInstances(boolean)` HOT 19
- How come TestNG listeners are global? HOT 3
- Failing while building testng version 7.7.1 using ./gradlew build HOT 3
- ListenerComparator doesn't work HOT 4
- ITestNGListenerFactory is broken and never invoked HOT 3
- can not set custom test method name when using data provider HOT 12
- Update JCommander to 1.83
- [Feature Request] allow multiple groups with AND condition HOT 2
- How to rerun skipped test cases using TestNG? HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from testng.